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1 METHOD AND APPARATUS FOR LOCALLY TARGETING VIRTUAL 

2 OBJECTS WITHIN A TERMINAL 

3 This application is a continuation-in-part of U.S. Application Serial Number 

4 09/597,893, filed June 19, 2000, entitled METHOD AND APPARATUS FOR 

5 TARGETING VIRTUAL OBJECTS, which is a continuation-in-part of U.S. Application 

6 SerialNumber09/054,419,filedApril3, 1998, entitled TARGETED ADVERTISEMENT 

7 USING TELEVISION DELIVERY SYSTEMS, which is a continuation-in-part of U.S. 

8 Application Serial Number 08/735,549, filed October 23, 1996, entitled METHOD AND 

9 APPARATUS FOR TARGETED ADVERTISING, which is a continuation of U.S. 

10 Application Serial Number 08/160,280, filed December 2, 1993, entitled NETWORK 

1 1 CONTROLLER FOR CABLE TELEVISION DELIVERY SYSTEM, now U.S. Patent 

12 Number 5,600,364, which was a continuation-in-part of U.S. Application Serial Number 

13 07/991,074, filedDecember 9, 1992, entitled REMOTE CONTROL FOR MENU DRIVEN 

14 SUBSCRIBER ACCESS TO TELEVISION PROGRAMMING. 

15 Technical Field 

1 6 The technical field relates to a method and apparatus for providing virtual objects that 

17 are targeted to subscribers based on processing performed locally by a subscriber' s terminal. 

18 ThememodandapparalusspecM and managing a content 

1 9 delivery network including an operations center, a local insertion center, or a subscriber's local 

20 terminal. 

21 Background 

22 Television as an advertising medium has undergone significant advances since its 

23 inception in 1939. Modern advances in digital technology now allow viewers to be very 

24 selectiveinchoosingprogramstowatch. Other advances in digital technology have led to such 

25 programming enhancements as a "tail" on a hockey puck, or an overlaid first down marker on 

26 a football field. The same technological advances allow improvements in the way 

27 advertisements are currently presented. Many sporting events are now presented with virtual 

28 object advertisements included in the video. Forexample, the broadcast of amajor league 
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1 baseball game may include one or more virtual object spots in which an advertisement is 

2 displayed. The advertisements are then seen by television viewers, but not by fans who attend 

3 the game. However, the advertisements are fixed, and are not varied according to individual 

4 viewers. 

5 Summary 

6 A system and a method delivers targeted virtual objects to reception sites. A virtual 

7 object is a realistic, synthetic replica of an actual object. The virtual object is viewable within 

8 video programming and may be combined with original video and audio to supplement or 

9 replace portions of the original video and audio content. Virtual objects may be overlaid on 

1 0 video, partially or entirely obscuring the underlying video. An overlaid object may be static in 

1 1 nature, such as a graphical icon or the like, or alternatively may be dynamic, such as a video 

1 2 clip, animation, or scrolling alphanumeric characters, for example. Overlaid objects may be 

1 3 limited spatially to a fixed portion of the video screen, limited temporally to a given time for 

14 display, limitedby a combination of both location and time, or tied to a spatially changing 

1 5 portion of the screen that is moving with time. Alternatively, virtual objects may be added to 

1 6 and embedded within the actual video. Multiple virtual obj ects may be embedded in the video 

17 in a multi-layer fashion. The virtual object is indistinguishable from the other video content 

1 8 sharing the field of view. Virtual objects may be interactive in nature. That is, a viewer may 

19 select an object and the selection will initiate a process whereby a reception site sends a 

20 command to a location designated by the interactive virtual object to initiate some action. 

21 An operations center may process the video signal to allow for the insertion of virtual 

22 objects into the video. An object delivery center serves as a standalone or supplemental 

23 system to the operations center to deliver virtual objects independently of the video with which 

24 the virtual objects are to be associated. A delivery network includes any of a number of 

25 different delivery systems to support the delivery of video and virtual objects from the 

26 operations center and the object delivery center to a local insertion center, or directly to a 

27 reception site. The delivery network is also used to deliver video and virtual objects from the 
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1 local insertion center to the reception site. The reception site receives the video and virtual 

2 objects and associates the appropriate virtual objects with the video based on targeting 

3 algorithms. 

4 The reception site collects virtual object viewing information. The reception site may 

5 use the virtual object viewing information and other information stored at the reception site to 

6 locally target the virtual obj ects at the reception site. Performing all targeting locally to the 

7 reception site maintains the subscriber' s privacy. The reception site also provides interactive 

8 requests, which are driven by the selection of interactive virtual objects, to an interactive object 

9 servicing center using the delivery network. Interactive responses are returned by the 

10 interactive object servicing center to the requesting reception site. 

1 1 The process of managing the content and the virtual objects to be included in the 

1 2 content begins with a number of configuration and set-up steps. Generic reception site and 

1 3 viewing population information can be collected at the operations center. This information is 

1 4 used to characterize the population of reception sites viewing the content in which virtual 

1 5 objects are to be placed. Information may be collected from various sources, including viewer 

1 6 surveys and marketing databases correlated by zip code+4, Nielsen or Arbitron program rating 

17 services, for example. 

1 8 Next, reception site groups are determined. For a number of target categories, groups 

19 are defined. Examples of target categories include demographic targeting (age/sex/income) and 

20 location, such as Area of Dominant Influence (ADI), for example. Each target category is then 

2 1 segmented into appropriate groups. For example, the ADI may include Los Angeles, CA and 

22 Washington D.C. New target categories can be added and the groups redefined after their 

23 initial establishment. 

24 Group assignment rules are then created by the operations center based on information 

25 being collected and tracked locally by each reception site. These group assignment rules are 

26 then periodically distributed to all reception sites. 
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1 For each target category, each reception site assigns itself to a group based on the 

2 information collected by the reception site and the group assignment rules provided by the 

3 operations center and stores the group assignments therein. 

4 The group assignment information that is stored at the reception site is able to survive 

5 power cycling of the reception site, and other normal service interruptions. Finally, as groups 

6 are modified or group assignment rules change, reception sites are notified of the changes using 

7 updated group assignment rules being distributed by an operations center. Additionally, the 

8 group assignment rules may be periodically resent to the reception sites to ensure that newly 

9 added reception sites and to reception sites that have accidentally lost their information are 

10 up-to-date. 

11 A virtual object location definer system determines where in the content the virtual 

1 2 objects are to be placed and the rules associated with placement of the virtual objects. Content 

1 3 may be video programming, commercials and advertisements, or electronic program guide 

1 4 (EPG) information, for example. A virtual object selector system determines which available 

1 5 virtual objects are suitable for placement in a virtual object location. A targeted virtual object 

1 6 management system determines which reception site groups should receive and display which 

1 7 virtual obj ect for a given virtual object location. The virtual obj ects and targeting information 

18 are then distributed to reception sites. 

1 9 After a reception site receives and stores the virtual objects and targeting information, 

20 the reception site will place the most appropriate virtual object into virtual object locations 

2 1 based on the targeting information, and will display the combined content with the overlaid or 

22 embedded virtual object. 

23 The reception site stores information indicating that a virtual objects was inserted. The 

24 accumulated history information may be collected by the reception site and stored for review 

25 purposes at a later time or for use with the group assignment rules. As mechanisms become 

26 available to identify specific viewers in a household, the reception site allows for individual 

27 identification information to also be provided with collected and stored data. 
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1 A centralized operations center can determine virtual object locations available for 

2 virtual object placement. Alternatively, a local insertion center can determine the virtual object 

3 locations. The operations center can determine the specific virtual objects to be placed in a 

4 virtual object location. Alternatively, the local insertion center may determine the specific virtual 

5 objects to be placed in a virtual object location. The reception site itself can determine which 

6 virtual object is to be placed in a virtual object location based on its own internal routines. 

7 Content, virtual objects, and associated targeting / virtual object placement control and 

8 group assignment rules can be relayed to reception sites. The reception site may reside within 

9 a digital cable set top box that has access to a delivery network. Alternately, the reception site 

10 may be components of digital television satellite receivers. The reception site may be 

1 1 incorporated into the circuitry of a television, thereby eliminating the need for a separate control 

1 2 device attached to the television. Alternatively, the reception site may be incorporated into a 

1 3 personal computer, personal data device, smart phone with a display, or electronic book 

14 device. 

15 Description Of The Drawings 

1 6 The detailed description will refer to the following drawings in which like numerals refer 

17 to like items, and in which: 

18 Figure 1 is an overview of the virtual object targeting delivery system; 

19 Figure 2 provides a pictorial representation of virtual objects and virtual object 

20 locations; 

21 Figure 3 is an example of an overlaid virtual object; 

22 Figure 4 is an example of an embedded virtual object; 

23 Figure 5 depicts an operations center; 

24 Figure 6 depicts a virtual object definer; 

25 Figure 7 is a pictorial representation of a virtual object location matte; 

26 Figure 8 depicts a virtual object selector; 

27 Figure 9 depicts a targeted virtual object management system; 
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1 Figure 10 shows configuration and set-up steps associated with targeting virtual 

2 objects; 

3 Figure 1 1 shows a subscriber information database system; 

4 Figure 12 shows a configuration set-up system; 

5 Figure 13 shows a virtual object targeting system; 

6 Figure 14 presents an embodiment of the overall process for assigning targeted virtual 

7 objects; 

8 Figure 1 5 presents an embodiment of a process used by the virtual object placement 

9 engine to assign virtual objects to virtual object locations; 

10 Figure 16presentsanaltemateembodimentusedbythevirUialobjectplacementengm 

11 to assign virtual objects to virtual object locations; 

12 Figure 17presentsyetanotherembodimentusedbythevta 

13 to assign virtual objects to virtual object locations; 

14 Figure 18 depicts an object delivery center; 

1 5 Figure 19 presents embodiments associated with the delivery of virtual objects over a 

16 coaxial or fiber cable system to a reception site; 

17 Figure 20 presents embodiments associated with the delivery of virtual objects over a 

18 wireless broadcast system to a reception site; 

1 9 Figure 2 1 presents embodiments associated with the delivery of virtual objects over a 

20 satellite broadcast system to a reception site; 

2 1 Figure 22 presents embodiments associated with the delivery of virtual objects over a 

22 wired data network to a reception site; 

23 Figure 23 presents embodiments associated with the delivery of virtual objects using 

24 the public switched telephony network (PSTN) to a reception site; 

25 Figure 24 presents embodiments associated with the delivery of virtual objects using 

26 wireless personal communications system (PCS) to a reception site; 
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1 Figure 25 depicts several embodiments associated with the delivery of virtual objects 

2 using a national or local television broadcaster's signal; 

3 Figure 26 depicts a local insertion center; 

4 Figure 27 depicts an example of a reception site; 

5 Figure 28 depicts an interactive object servicing center; and 

6 Figure 29 presents processing performed by an interactive object servicing 

7 center. 

8 Detailed Description 

9 An overview of the virtual object delivery system is depicted in Figure 1. An 

1 0 operations center 1 0 performs the processing of a video content signal to allow for the insertion 

11 of virtual objects into the content 36. An object delivery center 15 serves as a standalone or 

1 2 supplemental system to the operations center 1 0 to deliver virtual objects independent of the 

1 3 content with which the virtual objects are to be associated. A delivery network 1 1 includes any 

14 of a number of different delivery systems to support the delivery of the content 36 and virtual 

1 5 objects from the operations center 10 and the object delivery center 1 5 to a local insertion 

1 6 center 20 or directly to a reception site 30. A delivery network 1 2 is used to deliver content 

1 7 and virtual objects from a local insertion center 20 to the reception site 30. The reception site 

18 30 may be any device or terminal capable of receiving video, including a set top terminal, a 

1 9 television, a personal computer, a wireless telephone, a wired telephone, a PDA device, an 

20 electronic book, at digital satellite television receiver or any similar device or terminal. 

2 1 The reception site 30 receives the content 36 and virtual objects and associates the 

22 appropriate virtual objects with the content 36 based on internal targeting algorithms and group 

23 assignment rules. The reception site 30 may collect virtual object viewing information to be 

24 used in targeting virtual objects and in assigning the reception site to the appropriate group for 

25 each target category. The reception site 30 provides interactive requests, which are driven by 

26 the selection of interactive virtual obj ects, to an interactive object servicing center 60 using a 
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1 delivery network 14. Interactive responses are returned by the interactive object servicing 

2 center 60 to the requesting reception site 30. 

3 Virtual objects may be realistic, synthetic replicas of actual objects. Virtual objects 

4 may also be caricatures of actual individuals, photographs or other life-like renderings of actual 

5 individuals, cartoon figures, text objects, graphical renderings, or icons, for example. The virtual 

6 objects maybe animated or fixed. The virtual objects are combined with video and audio to 

7 supplement or replace portions of video and audio in original content 36. As shown in Figure 

8 2, the reception site 30 may contain or be connected to a display 35 on which the content 36 

9 may be displayed. An opportunity, advertisement spot, or location, in the content 36 that is 

10 available for the placement of the virtual object will be denoted as a virtual object location 37 

1 1 henceforward. Within the virtual object location 37, one or more individual virtual objects may 

12 be assigned, each denoted as a virtual object 38 henceforward. Multiple virtual object 

13 locations, shown as virtual object locations 37 and 39 may be present in the content 36. 

14 Multiple virtual objects, shown as virtual objects 38 and 40 may be present within the virtual 

15 object locations. 

16 As shown in Figure 3, virtual objects may be overlaid on video, partially or entirely 

17 obscuring the underlying video. An overlaid virtual object may be static in nature, like a 

1 8 graphical icon, as shown by virtual object 42. Alternatively the overlaid virtual object may be 

1 9 dynamic, like a video clip, animation, or scrolling alphanumeric characters as shown by virtual 

20 object 44. Overlaid virtual objects maybe limited spatially to a fixed portion of the video, 

2 1 limited temporally to a given time for display, or limited by a combination of both location and 

22 time. Overlaid virtual objects may also be tied to a spatially changing portion of the video that 

23 is moving with time. 

24 Alternatively, as shown in Figure 4, virtual objects may be added to and embedded 

25 within video, hi this alternative, a synthetic virtual object could be indistinguishable from the 

26 other video content 36 sharing the field of view as shown by virtual object 46 and virtual object 

27 48 . For instance, today's technology allows for the virtual placement of a billboard at televised 
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1 sports events and the placement of a virtual first down marker in televised football 

2 games. 

3 In an embodiment, virtual reality and animation technologies are combined with 

4 advanced digital video techniques to provide realistic interaction of virtual objects within video. 

5 Combining these technologies, a soda can may be synthetically placed in the video, and may 

6 then be made to change over time. This placement and subsequent modification can occur at 

7 the video's source, at an intermediate point within the distribution and delivery path, or at the 

8 reception site 30. Combining the placement of virtual objects with the ability to target specific 

9 virtual obj ects to specific viewers or groups of viewers allows one household to see a scene 

10 with the soda can for cola, while the next door neighbor sees a root beer soda can, for 

1 1 example. 

1 2 Virtual objects may be interactive in nature, where a viewer can select a virtual object 

13 35 and this selection will initiate a process whereby the reception site 30 sends a command to 

14 the location designated by the interactive virtual object 38 to initiate some action. Actions may 

15 include linking to a Web site to display content related to the interactive virtual object 38, 

1 6 initiating a purchase transaction, or initiating a request for more information about the selected 

17 virtual object 38. 

1 8 The operations center 1 0 shown in Figure 1 may include a number of systems that act 

1 9 together in processing the content 36 for the inclusion of virtual objects, for the selection of 

20 appropriate virtual objects to be placed in the content 36, for the targeting of virtual objects to 

2 1 reception site groups, and for the packaging and delivery of the content 36 and virtual objects 

22 to reception sites. 

23 Placement of virtual objects can be explicitly selected by the operations center 10, 

24 resulting in the specific selection and placement of virtual objects into content 36. Alternatively, 

25 the placement maybe generically defined by the operations center 10. In this alternative, the 

26 reception site 30 performs all the processing associated with selecting the appropriate virtual 
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1 object 38 to be placed in the content 36 based on basic guidelines provided by the operations 

2 center 10 and algorithms operating at the reception site 30. 

3 As shown in Figure 5, the operations center 10 includes a virtual object location definer 

4 100, a virtual object selector 200, and a targeted virtual object management system (TVOMS) 

5 300. 

6 Figure 6 presents the virtual object location definer 100. A video capture processor 

7 1 10 processes video and audio content 36 on a frame by frame basis, converting the original 

8 content 3 6 into a corresponding digitized representation. The processed content 36' is then 

9 stored in content buffer 1 20 for future access. A pre-viewer subsystem 130 allows for the 

1 0 viewing of a video frame of the processed content 36'. Frame N 14 1 , for example, (shown in 

1 1 Figure 7) associated with the processed content 36', may be retrieved from the content buffer 

12 120, viewed, and passed to a location selector processor 140. The location selector processor 

13 140 allows for the selection ofwhere in the frame N 141 the virtual object 38 may be placed. 

14 When the frame N 141 is retrieved by the location selector processor 140, either a static area 

1 5 may be selected, or alternatively, a dynamic area, which is tied to an area within the frame of 

16 the processed content 36', maybe selected. An overlay matte 16 (see Figure 7) may be used 

17 in the virtual object insertion process to identify where and how a virtual object location 37 is 

18 to be placed in the processed content 36'. 

19 Techniques for pattern recognition used by the location selector processor 140 to 

20 facilitate the creation of the matte 1 6 and the identification of the pixels within the frame that the 

2 1 matte 1 6 is to be associated with for that frame are described in detail in US Patent 5,808,695, 

22 to Rosser, Roy J. ; Das, Subhodev; and Tan, Yi; entitled Method of Tracking Scene Motion 

23 for Live Video Insertion; US Patent 5,903,3 17, to Sharir, Avi; and Tamir, Michael; entitled 

24 Apparatus and method for Detecting, Identifying, and Incorporating Advertisements in a Video; 

25 US Patent 5,524,065, to Yagasaki, Toshiaki; entitled Method and Apparatus for Pattern 

26 Recognition; US Patent 5,627,915, to Rosser, Roy J.; Das, Subhodev; and Tan, Yi; von 

27 Kaenel, Peter; entitled Pattern Recognition System Employing Unlike Templates to Detect 
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1 Objects Having Distinctive Features in a Video Field; and US Patent 4,8 17, 17 1 , to Stentiford, 

2 Frederick; entitled Pattern Recognition System, the disclosures of which are hereby 

3 incorporated by reference. 

4 When the area is selected by the location selector processor 140 and the overlay matte 

5 1 6 for the initial video frame N 14 1 is created, a video object marker processor 1 60 creates 

6 the transparent overlay matte 16 that is associated with the selected area for subsequent 

7 frames, for example frame N+l 142 and frame N+2 143 ofthe processed content 36 ? , for the 

8 duration of frames designated, as shown in Figure 7. This selected area defines the virtual 

9 obj ect location 37 . Pattern recognition technology may then be applied to each subsequent 

10 frame of the processed content 36' in the video object marker processor 160, creating a 

1 1 sequence of mattes to be applied to each frame of the processed content 36', moving and 

1 2 transforming as needed to match the temporal movement and transformations ofthe virtual 

1 3 object location 37 within the processed content 36'to which the virtual object 38 is to be tied. 

14 The pattern recognition technology handles transitions, cutaways, and cutbacks within the 

1 5 processed content 36', and any visual blocking or occlusions that may occur as other objects 

1 6 within the processed content 36' appear in front of the dynamic area selected for virtual object 

17 location 37. 

1 8 Simultaneously with the selection of the virtual object location 37 and the creation of 

19 the mattes, a virtual object rules processor 170 allows for the entry of rules that govern the 

20 types of virtual objects and other relevant placement guidelines associated with the virtual 

2 1 object location 37. These rules allow for the selection of characteristics such as the duration 

22 ofthe virtual object location 37, and viewing overlay characteristics such as transparency of 

23 the overlay virtual object. The operations center 10 processes the stored, non-realtime 

24 processed content 36' and the real-time (live) processed content 36'. For real-time processed 

25 content 36', the content buffer 120 serves as a short buffer, and predefined rules are 

26 pre-loaded into the virtual object rules processor 170. Additionally, the video object marker 

27 processor 1 60 is pre-loaded with the directions as to which locations within the processed 
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1 content 36'are to be treated as virtual object locations. The video object marker processor 

2 1 60 then automatically searches the real-time processed content 36'using pattern recognition 

3 technologies presented above, or other technologies, and automatically creates the mattes 

4 required for each virtual objectlocation. Once the video object marker processor 160creates 

5 the mattes and the associated controls, the mattes are associated with the actual processed 

6 content 36'in the content buffer 120. The processed content 36', along with the mattes are 

7 then optionally processed using the optional video processor 150, which performs any 

8 necessary content encoding (e.g., MPEG4, or digitalization), and makes the content 36' 

9 available to a rules application processor 1 80. The rules application processor 1 80 creates 

10 metadata packets that carry the virtual object placement rules information and mattes and 

1 1 associates these packets with the processed content 36' for each virtual object location 37 

12 selected in the virtual object location definer 100. 

1 3 Figure 8 is a block diagram of the virtual object selector 200. Processed content 36', 

14 along with the metadata packets carrying the virtual object placement rules information 

1 5 associated with each virtual obj ect location 37 and the mattes 1 6 are provided by the virtual 

16 object location definer lOOtothe virtual object selector 200. An object selector processor 

17 210 extracts the placement rules and stores the processed content 3 6 ' in a content buffer 240 . 

1 8 Using the placement rales, along with any operator entered object placement guidance, the 

1 9 object selector processor 210 queries an object matcher processor 230 to initiate the selection 

20 of virtual objects that match the requisite rules. The object matcher processor 230 can be 

2 1 commanded by the object selector processor 2 10 to match a virtual object 38 in at least three 

22 manners: 1) automatically, 2) with manual placement, and 3) with pre-selected virtual objects. 

23 For automatic matching, the object matcher processor 230 searches an available virtual objects 

24 database 220 to find virtual objects that meet the placement rules provided by the object 

25 selector processor 210. The matching virtual objects are then marked in the available virtual 

26 objects database 220 as suitable for that virtual object location 37. For manual matching, the 

27 operator of the object matcher processor 230 manually selects the desired virtual objects to 
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1 be associated with a virtual object location 37, and marks the selected virtual objects as 

2 suitable for the virtual object location 37 in the available virtual objects database 220. For 

3 pre-selected objects, the placement rules will indicate the pre-defined virtual objects to be 

4 associated with the processed content 361 The object matcher processor 230 marks the 

5 pre-determined virtual objects in the available virtual objects database 220 as being associated 

6 the particular processed content 36' and virtual object location 37. 

7 Virtual objects may be processed and stored in the available virtual objects database 

8 220 before they are used. Processing of the virtual objects includes digitizing the virtual object 

9 38 and associating the virtual object with those virtual object 38 placement guidelines and rules 

10 that mustbe followed to place the virtual object 38 within virtual object locations. The rules 

1 1 and guidelines may include product categories with which the virtual object 38 should be 

1 2 associated, or in contrast, cannot be associated with, the type of virtual object 3 8, the duration 

1 3 that the virtual object 38 is valid to be used, and the number of times the virtual object 38 may 

14 be used. 

15 In a non-realtime environment, an optional post viewer processor 260, which is 

1 6 preceded by a virtual object insertion processor 250, is used to view the content 36 and insert 

1 7 each virtual object 38 that was matched to the content 36 by the object matcher processor 230 

18 in the corresponding virtual object location 37. Techniques for insertion of overlaid virtual 

19 objects are described in detail in U.S. Patents 4,3 19,266 to Bannister, Richard S.; entitled 

20 Chroma Keying System; 4,999,709 to Yamazaki, Hiroshi; and Okazaki, Sakae; entitled 

21 Apparatus for Inserting Title Pictures; 5,249,039, to Chaplin, Daniel J.; entitled Chroma Key 

22 Method and Apparatus; and 5,233,423 to Jernigan, Forest E. ; and Bingham, Joseph; entitled 

23 Embedded Commercials within a Television Receiver using an Integrated Electronic Billboard, 

24 the disclosures of which are hereby incorporated by reference. 

25 Techniques for the insertion of embedded virtual objects are described in detail in U.S . 

26 Patents 5,953,076, to Astle, Brian; and Das, Subhodev; titled System and Method of Real 

27 Time Insertions into Video Using Adaptive Occlusion with a Synthetic Reference Image; 
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1 5,892,554, to DiCicco, Darrell; and Fant, Karl; entitled System and Method for Inserting 

2 Static and Dynamic Images into a Live Video Broadcast; 5,5 15,485, to Luquet, Andre; and 

3 Rebuffet, Michel; entitled Method and Device for Modifying a Zone in Successive Images; 

4 5,903,3 17, to Sharir, Avi; andTamir, Michael; entitled Apparatus andMethod for Detecting, 

5 Identifying and Incorporation Advertisements in a Video; and the MPEG4 standard, the 

6 disclosure of which are hereby incorporated by reference. 

7 In a realtime environment, the optional post viewer processor 260 is bypassed, and the 

8 default virtual object 38 is placed in the virtual object location 37 by a default virtual object 

9 insertion processor 270, which includes (not shown) a virtual obj ect insertion processor 250. 

10 The targeted virtual object management system (TVOMS) 300 shown in Figure 9 

1 1 allows for virtual objects, including virtual object-based advertisements, to be directed to 

1 2 groups of subscribers based on, for example, the use of subscriber data, programs watched 

1 3 data, past virtual objects viewing data, and/or mood indicators entered by the subscriber. 

1 4 Alternatively, input from subscribers collected through form-based on screen questionnaires 

1 5 may be used to further define a subscriber's potential likes, wants, and needs and store this 

16 information at the reception site 30. Advertisers wanting to optimize their advertising 

1 7 expenditures may direct virtual objects to the appropriate viewing audiences to ensure that the 

1 8 desired audience views specific virtual objects. Advertisers can display specific virtual objects 

19 in content 3 6 that is being viewed by those subscribers most likely to be influenced to buy the 

20 advertised product, or otherwise respond in a desired fashion to the virtual objects. 

2 1 Virtual objects may also be targeted to reception sites on various levels. At a highest 

22 level, virtual objects can be delivered to all reception sites viewing content 36, with no targeting 

23 of the virtual objects to the subscriber, but with the virtual objects displayed in the content 36 

24 that are determined to be most relevant to the content 36. That is, the virtual objects are 

25 placed in the virtual obj ect location 37 without the use of any group targeting. Alternatively, 

26 some level of targeting may occur based on, for example, ADI, zip code +4, geographical data 

27 and other similar criteria known by the reception site 30. In this alternative embodiment, the 
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1 virtual objects are sent to a reception site, and a local insertion routine at the reception site 30 

2 controls placement of the virtual objects into the virtual object locations 37 in the content 36. 

3 The virtual objects maybe stored at the reception site 30 and maybe periodically refreshed. 

4 To account for reception sites that do not have virtual obj ects available for insertion, the content 

5 36 may be provided with a default virtual object 38 embedded in the content 36. Upon receipt 

6 of the content at a reception site 30, the reception site 30, using the local insertion routine, 

7 determines if the default virtual object 38 should be replaced with another virtual object residing 

8 in the reception site's memory or being delivered concurrently with the content 36. 

9 Alternatively, virtual objects may be targeted to groups of reception sites, with the 

1 0 groups of reception sites categorized based on some other common subscriber characteristics 

1 1 such as programs watched data, for example. Finally, virtual objects may also be targeted to 

1 2 specific subscribers that share the use of a reception site 30 based on their unique subscriber 

1 3 characteristics. Individual subscribers sharing the use of a reception site 30 may identify 

14 themselves when beginning to view content 36 at the reception site 30, resulting in the unique 

1 5 assignment of groups for the reception site 30 during that individual subscriber' s use of the 

16 reception site 30. 

1 7 The TVOMS 300 provides the management of information required to support each 

1 8 of the following: (1) delivery of targeted virtual objects along with content 36 being broadcast; 

19 (2) delivery of targeted virtual objects to reception sites independent of any content 36 being 

20 broadcast; and (3) delivery of TVOMS-related information including group assignment rules 

21 and retrieval plans to reception sites. The TVOMS 300 may be executed in software routines 

22 operating on a general purpose computer or a processor. The TVOMS 300 may also be 

23 executed, at least partly, in hardware. Finally, the TVOMS 300 may include various databases 

24 containing information to support targeting of virtual objects. 

25 Figure 9 shows the TVOMS 300 supporting the targeting of virtual objects to groups 

26 of subscribers. Broadcast information can be destined for the entire population of subscribers 

27 receiving the content 36, or groups of subscribers. Broadcast information can include actual 
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1 content 36, metadata packets with virtual object insertion control information, virtual objects 

2 for placement within the content 36, and group assignment rules required by the subscriber's 

3 reception site 30 to configure the reception site 30 and retrieval plans to guide the reception 

4 site 30 in placing the appropriate virtual object 38 within the content 36. Broadcasting of this 

5 information may be supported over a variety of broadcast-capable communication systems, 

6 such as the Internet, cable television systems, terrestrial broadcast systems, satellite broadcast 

7 systems, and wireless communications systems, and other systems described below. 

8 A viewing population information database 1 2 10 in the TVOMS 300 contains viewing 

9 population information collected from numerous sources including marketing database services, 

10 from Arbitron or Nielsen program rating services, or subscriber surveys, for example. The 

1 1 viewing population information may be used by a virtual object targeting system 1 220 to 

1 2 determine the best virtual obj ects to be distributed for inclusion in the content 3 6. The virtual 

1 3 object targeting system 1 220 determines the optimum subset of virtual objects to be associated 

14 with the content 36 based on the selected object metadata provided by the virtual object 

1 5 selector 200 (Figure 5) and viewing population information from the viewing population 

16 information database 1210. A content and virtual object packager 1260is directed to retrieve 

17 the appropriate virtual objects from an available virtual objects database 1265. Thecontent 

18 and virtual object packager 1260, then, along with the content 36 from a content buffer 1270, 

1 9 addresses the virtual objects with the appropriate group addressing information, and packages 

20 the virtual objects with the content 36. A delivery processor 1 300 then delivers the combined 

21 package of virtual objects, content 36, and metadata to subscribers. 

22 As an alternative to delivering virtual objects with associated content 36, virtual objects 

23 can be delivered independently to groups of subscribers as a result of modified group 

24 assignments or group assignment rules , or the need for refreshed virtual objects at reception 

25 sites. Initiation could be automatic based on a scheduled cycle or by TVOMS operator 

26 direction. Upon delivery initiation, the virtual object targeting system 1220 uses viewing 

27 information from the viewing population information database 12 10, information about available 
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1 virtual objects from the available virtual objects database 1265, and information about 

2 previously delivered virtual objects from the viewing population information database 12 10, to 

3 select the appropriate virtual objects to be packaged and delivered to a group of reception 

4 sites. Once the virtual object targeting system 1 220 determines the appropriate virtual objects, 

5 the content and virtual object packager 1260 retrieves the appropriate virtual objects, 

6 packages the virtual objects, addresses the information to a group or groups of reception sites, 

7 and delivers the information to the appropriate groups of reception sites using a delivery 

8 processor 1300. Virtual objects may be broadcast to all reception sites, and a reception site 

9 30 may store only the virtual objects that are associated with groups to which the reception site 

10 30 belongs. Alternatively content 36, virtual objects, and other information destined to 

1 1 reception sites may be provided to the object delivery center 15 (Figure 1) for delivery to 

12 reception sites. 

13 The databases addressed in Figure 9 may be configured to support a variety of 

14 information necessary for the TVOMS 300 to manage the targeting process. Below are tables 

15 that present typical data that may be tracked by these individual databases. 

16 Viewing Population Information Database 1210 

17 Criteria being captured by reception sites for targeting purposes 

18 Target categories and groups for each category 

19 Current group assignment rules 

20 Past group assignment rules 

2 1 Upcoming group assignment rules 

22 Estimated number of reception sites per zip+4 area 

23 Estimated number of reception site subscribers by: 

24 Sex 

25 Age 

26 Education 

27 Profession 
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1 Household income 

2 ADI 

3 Past subscriber survey data or actual data presenting, by subscriber group type: 

4 TV program preferences 

5 Demographic information 

6 Past virtual object viewed data, time spent viewing 

7 Past products ordered, along with time, date, and method of order 

8 Past TV programs watched data, along with time and date 

9 Past PPV programs ordered data, along with time and date 

10 Nielsen and Arbitron program viewing statistics 

1 1 Generic subscriber profile information. 

1 2 for each category 

13 Available Virtual Objects Database 1265 

14 Virtual object identifier with actual digital version of virtual (CR) object 

15 Display options (e.g., text, audio, graphics, video, link, HTML, XML, interactive) 

16 Static vs. dynamic virtual object indicator, 

17 If a linked virtual object, link table information 

1 8 Pricing subsidy information 

19 Run through completion status mode indication 

20 Date of valid use 

21 Virtual object placement controls, acceptable frequency 

22 Category and group preferences (as virtual object ranking percentages) 

23 Within the TVOMS 300, the virtual object targeting system 1220 is responsible for the 

24 intelligent and rapid selection of virtual objects for placement in content 36. Category and 

25 group targetingismanagedinamannersimilarto that described in co-pending U.S. Application 

26 Serial No. 09/597,893, entitled METHOD AND APPARATUS FOR TARGETING 

27 VIRTUAL OBJECTS, filed June 19, 2000, and in co-pending U.S . Application Serial No. 
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1 09/054,419 entitled TARGETED ADVERTISEMENT USING TELEVISION DELIVERY 

2 SYSTEM, filed April 3, 1998, and in co-pending U.S. Application Serial No. 09/328,672 

3 entitled ELECTRONIC BOOK SELECTION AND DELIVERY SYSTEM WITH 

4 TARGETED ADVERTISING, filed on June 9, 1 999, each of which are incorporated herein 

5 by reference. 

6 Careful management of the virtual objects within the content 36, based on information known 

7 about the demographics and viewing habits of typical subscribers, for example, can greatly 

8 increase both the advertisers' likelihood of reaching an interested subscriber, and the likelihood 

9 a subscriber will be interested in a specific virtual object 3 8 . Each virtual object location 37 

10 within the content 36 is assigned a series of virtual objects by the TVOMS 300, and when 

1 1 multiple virtual objects are delivered for a given virtual object location 37 in the content 3 6, a 

1 2 retrieval plan is developed that directs which virtual objects should be displayed by a given 

13 group of subscribers or reception sites, or the entire subscriber population. 

14 The process of managing the targeted virtual objects may include a number of 

1 5 configuration and set-up steps shown in Figure 10 that begins with the start step shown in block 

16 7010 and ends with the end step shown in block 7017. First, a number of target categories 

17 are defined as shown in block 70 1 3 . Examples of target categories include demographic 

1 8 targeting (age/sex/income) and location, such as Area of Dominant Influence (ADI) . Next, as 

1 9 shown in block 7014, each target category may be segmented into appropriate groups. For 

20 example, the ADI may include the groups Los Angeles, CA and Washington D.C. New target 

2 1 categories can be added and the groups included in the target category redefined after their 

22 initial establishment. 

23 Next, as shown in block 7015, for each target category, group assignment rules are 

24 created that allow for reception sites to be assigned to a group for each target category based 

25 on the information collected at the reception site 30. The group assignment rules generation 

26 process is described in detail later. Group assignment rules are conveyed to all reception sites 

27 and are stored therein, as shown in block 70 16. As groups are modified or group assignment 
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1 rules change, the reception sites are provided with the changes. Additionally, the group 

2 assignment rules are periodically broadcast to ensure that newly added reception sites and 

3 those reception sites that have accidentally lost their information are up-to-date. As an 

4 alternative to, or in conjunction with conveying the group assignment rules, an initial set of group 

5 assignment rules may be provided with the reception site 30. For example, if the reception site 

6 30 is a television set top terminal, the terminal may have embedded as a software routine, an 

7 initial set of group assignment rules. The group assignment rules may then be updated as 

8 needed. 

9 Returning to Figure 9, the virtual object targeting system 1220 determines the optimum 

1 0 types of virtual objects to be placed in the content 36 from the selected virtual objects provided 

11 by the virtual object selector 200 (Figure 5). The virtual object targeting system 1220 takes 

1 2 into account groups of subscribers who will likely view the content 36, the desirability of 

1 3 providing available virtual objects to those groups of subscribers, target categories, the number 

14 of virtual objects locations available for the content 36, and the number of virtual objects 

15 available for assignment for a given virtual object location 37. 

1 6 Once specific virtual objects are selected for one or more available virtual object 

17 locations, the groups that should view each virtual object 38 are determined, based on the 

1 8 target category of interest. For example, in target category ADI, and group Los Angeles, CA, 

19 a virtual object related to a Los Angeles area professional basketball team may be considered 

20 appropriate as an advertisement for professional basketball while in the group Washington, 

21 D.C., a virtual object related to a Washington, D.C. professional basketball team might be 

22 considered appropriate. The selected virtual object locations 37 may include all virtual object 

23 locations, or a subset of all the virtual object locations . A retrieval plan is generated by the 

24 retrieval plan generator 1 275 which provides information concerning which target category and 

25 group is assigned to each virtual object 38 associated with each virtual object location 37. The 

26 retrieval plan may provide information for one virtual object location 37 or multiple virtual 

27 object locations within content 36, where one or more virtual objects, target categories, and 
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1 the groups to which each virtual object 38 is targeted within each virtual object location 37 is 

2 also provided in the retrieval plan. An example retrieval plan is provided in Table C below. 

3 Retrieval plans may be distributed along with the virtual objects and the associated content 36 

4 directly to the reception sites by the delivery processor 1 300 or using the object delivery center 

5 15. Alternatively, a retrieval plan may be distributed by the delivery processor 1 300 or using 

6 the object delivery center 15 independent of the associated content 36 or virtual objects. 

7 After the reception site 30 receives and stores the virtual objects and the retrieval plan, 

8 the reception site 30 inserts those virtual objects into the appropriate virtual object locations 

9 in the content 3 6 based on the retrieval plan. The reception site 30 may retrieve and store only 

1 0 those virtual objects associated with that reception site's group assignment for that virtual object 

1 1 location 37 . Alternatively, the reception site 30 may retrieve and store all virtual objects but 

12 only insert those virtual objects into virtual object locations as dictated by the retrieval 

13 plan. 

14 When the virtual objects are displayed within the content 36, the reception site 30 will 

15 store virtual objects viewed data indicating that a virtual object 38 was shown. In an 

1 6 embodiment, the reception site 30 will store this virtual object viewed data only if the virtual 

17 objects are displayed for a predetermined time, or only if the subscriber takes an action to 

1 8 indicate the virtual object 38 has been viewed, such as by selecting an interactive virtual object 

19 38, for example. Accumulated virtual objects viewed data may be used by a reception site 30 

20 in conjunction with the group assignment rules. 

2 1 The virtual obj ect targeting system 1 220 receives requests from the metadata extractor 

22 processor 1200 to initiate the determination of virtual objects to be placed. The metadata 

23 extractor processor 1 200 receives content 3 6 and associated virtual object information from 

24 the virtual object selector 200 (Figure 5). The virtual object targeting system 1220 provides 

25 outputs to the content and virtual object packager 1260 and the retrieval plan generator 1275. 

26 A part of the TVOMS 300 operation is the collection of viewing population data, and 

27 the assimilation of the viewing population data into the virtual objects selection method. This 
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1 operation typically includes two steps. First, viewing population data is retrieved from various 

2 sources. The viewing population data is then compiled and sent to the viewing population data 

3 collection engine 1 202 in the operations center 10. Once assembled, the data is filtered for 

4 each application of the TVOMS 300. In an embodiment, the viewing population information 

5 database 12 10 receives inputs from the viewing population data collection engine 1202anda 

6 configuration set-up system 1205. The viewing population information database 1210provides 

7 outputs to the configuration set-up system 1205, and the virtual object targeting system 1220. 

8 In addition to demographic information gathered, the viewing population information 

9 can be compiled using other methods. For instance, viewing population information can be 

1 0 gathered using questionnaires sent by mail to subscribers or collected over the telephone from 

1 1 subscribers and subsequently entered in the viewing population information database 1210. 

1 2 Subscriber preference information may be collected in surveys, including information such as 

1 3 sex, age, place of birth, place of lower school education, employment type, level of education, 

14 amount of television program viewing per week, and the number of television shows in 

15 particular categories that the subscriber watches in a given week such as, sports, movies, 

1 6 documentaries, sitcoms, amount of Internet use and favorite web sites, etc. Any demographic 

1 7 information that will assist the TVOMS 300 in targeting virtual objects may be collected. The 

18 viewing population data collection engine 1202 may gather marketing information from various 

1 9 sources and index the information for inclusion in the viewing population information database 

20 1210. 

21 To maintain the viewing population information database 1210 within the TVOMS 300, 

22 a database server 1190, communications server 1191, subscriber workstation 1192 or 

23 stations, or the suitable equivalents thereof, may be used, as depicted in Figure 1 1 . The 

24 database server 1 1 90 supports saving database files, event logging, event scheduling, database 

25 server services, and database security access. 

26 The communications server 1191 performs the following functions on database data: 

27 integrity check, filtering, processing, and uploading viewing population data from external 
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1 sources using the viewing population data collection engine 1 202. The workstation 1192 

2 allows for operator viewing and entry of viewing population data into the viewing population 

3 information database 1210. 

4 Figure 12 shows an example of the configuration set-up system 1205 in more detail. 

5 An interface 1206 receives viewing population information. The interface 1206 can include a 

6 workstation, such as the workstation 1 209, for example, from which an operator manually 

7 enters viewing population information. Alternately, viewing population information can be 

8 automatically entered at the interface 1 206 by downloading from an off-site database, the 

9 Internet, a storage medium, such as a CD-ROM or a floppy disk, or by collecting the 

1 0 information directly from the external sources using the viewing population data collection 

1 1 engine 1 202. A processor 1 207 processes the received viewing population information and 

12 organizes the information for use. For example, the processor 1207 may create a 

1 3 Category/Group Definition Matrix as presented in Table A and generate the group assignment 

14 rules that can be used to target virtual objects to groups of reception sites. The 

1 5 Category/Group Definition Matrix and group assignment rules will be described in more detail 

1 6 later. The Category/Group Definition Matrix and group assignment rules and organized viewing 

17 population information are then stored in a database 1208, and are periodically updated as 

18 viewing population information or group assignment rules, for example, change. 

1 9 The information gathered by the configuration set-up system 1 205 can come from a 

20 variety of sources including marketing databases, survey inputs from the subscribers, data 

2 1 collected by the viewing population data collection engine 1202, and other sources. The 

22 processor 1 207 will assign category numbers to target categories. For example, the ADI could 

23 be assigned category 1 and household (HH) income could be assigned category 2. Next, the 

24 configuration set-up system 1205 creates a number of non-overlapping groups for each 

25 category. For example, ADI can be broken down into Seattle, WA, Washington D.C., 

26 Denver CO. , Los Angles C A, etc. Similarly, HH income can be broken down into a number 

27 of income groups such as no income, 20-40K, 40-60K, 60- 120K, and over 120K. Then, the 
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1 configuration set-up system 1205 assigns a "group mask representation" for each group within 

2 every category. The group mask representation may be simply a binary number that can be 

3 used to identify a particular group. Table A shows a completed Category/Group Definition 

4 matrix that could be used by the virtual obj ect targeting system 1 220 to assign targeted virtual 

5 objects to groups of reception sites. 

6 Table A - Category/Group Definition Matrix 



11 



Category 
Number 


Category 
Name 


Group 
Number 


Group Definition 


Group Mask 
Representation 






1 


Seattle, WA 


1000000000 


1 


ADI 


2 


Washington, DC. 


0100000000 






3 


Denver, CO 


0010000000 






4 


Los Angeles, CA 


0001000000 






1 


No income 


1000000000 


2 


HH income 


2 


20-40K 


0100000000 






3 


40-60K 


0010000000 






4 


60-120K 


0001000000 






1 


Group a 


1000000000 






2 


Group b 


0100000000 


3 


Category x 


3 


Group c 


0010000000 






4 


Group d 


0001000000 






5 


Group e 


0000100000 






6 


Group f 


0000010000 



12 The processor 1207 also generates the group assignment rules for distribution to 

1 3 reception sites. The group assignment rules allow for the assignment of a group number to 

1 4 each reception site 30 for each target category based on information collected and stored at 

1 5 the reception site 30. The information used in conjunction with the group assignment rules is 

1 6 maintained in a subscriber database within the group assignment rules processor 735 at the 

1 7 reception site 30. Each reception site 30 populates its subscriber database through a variety 

18 of means, including subscriber input via a user interface at the reception site 30. Subscriber 
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1 inputs may be driven by reception site prompts upon receipt of group as signment rules from the 

2 operations center 10 or other on-screen questionnaires generated by the reception site 30. The 

3 subscriber database may be populated automatically by the reception site 30 as events take 

4 place at the reception site such as programs are watched by the subscriber or selected for 

5 recording, virtual objects are viewed by the subscriber, PP V events are purchased, interactive 

6 virtual objects are selected, or based on Internet usage. Alternatively, upon installation of the 

7 reception site 30, an installer or the subscriber may complete an installation process that results 

8 in the subscriber database being populated. Additionally, the reception site 30 may contain, 

9 or interface to, a Global Positioning Satellite (GPS) receiver that may provide accurate 

1 0 reception site location information that is stored in the subscriber database for use in locality- 

1 1 based targeting. Subscriber database information may also come from external sources such 

12 as downloads from the Internet or, from the operations center 1 0 or local insertion center 20 

1 3 using group assignment rules. Information contained in the subscriber database at the reception 

14 site 30 may include the following: 

15 Reception system identification information 

1 6 Reception site type 

17 Date of system set-up 

1 8 Household income 

19 User data (for each registered subscriber), including: 

20 Name 

21 Sex 

22 Age 

23 Place of birth 

24 Education 

25 Profession 

26 TV program preferences 

27 Demographic information 
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1 Past virtual objects viewed data, number of times viewed, time spent viewing 

2 Past products ordered, along with time, date, and method of order 

3 Past billing information 

4 Past TV programs watched data, along with time and date 

5 Past PPV programs ordered data, along with time and date 

6 Mood indicators entered at time of reception site turn on 

7 Form-based questionnaire results from subscriber 

8 Group assignments per subscriber for each target category 

9 Past virtual objects delivered to subscriber, date of delivery, method of delivery 

10 Zip+4 information 

1 1 Household location or address 

12 Number of subscribers in household 

13 Responses to prompts from past group assignment rules interactions 

1 4 Group assignment rules can be distributed that manipulate the subscriber database for 

1 5 example by adding an additional field or additional fields to the subscriber database at reception 

1 6 sites. If the new field requires the subscriber to provide information and the reception site 30 

17 is capable of prompting the subscriber and the subscriber is willing to provide the information, 

18 the reception site 30 will prompt the subscriber to enter the necessary information required for 

1 9 the newly added field or fields of data. If the new field or fields require the reception site 30 

20 to begin automatically tracking new information to populate the field, the group assignment rule 

2 1 processor 735 will begin to collect the requisite information at the reception site and store the 

22 information in the subscriber database at the reception site 30. 

23 Group assignment rules may also be used to place values in a subscriber database field 

24 or subscriber database fields based on the contents of another subscriber database field or 

25 other subscriber database fields. 



-26- 



Docket 5220/PTO Filings/Spec, wpd 

1 Group assignment rules may also be used to assign a reception site to a group within 

2 a target category based on the value of a subscriber database field or combination of subscriber 

3 database fields. 

4 What follows below is an embodiment of the group assignment rules used to associate 

5 reception sites to groups for target category X using as an example the Category / Group 

6 Definition Matrix presented in Table A. As an example, a group assignment rule may be 

7 created and distributed that results in the subscriber database being configured to track the sex 

8 of the subscriber (male or female) and the profession of the subscriber (doctor, lawyer, 

9 construction worker, college professor, teacher, or other). The delivery of this group 

1 0 assignment rule information may result in the subscriber at the reception site 30 being prompted 

11 to respond via an onscreen menu as to the subscriber' s sex and profession. If this information 

12 is already stored in the subscriber database as a result of prior interactions, no prompt of the 

1 3 subscriber would be initiated by the reception site 30. Alternatively, agroup assignment rule 

14 maybe created and distributed that uses information automatically collected at the reception 

1 5 site 30, such as programs watched information, to impute a sex and profession for a reception 

1 6 site 30 based on the distributed group assignment rules. For example, research may indicate 

1 7 that women are more likely to watch professional billiards, so a group assignment rule can be 

1 8 created and distributed that dictates that if a reception site 30 has viewed any professional 

1 9 billiards, then the sex of the subscriber can be set to "female". Similarly, if research indicated 

20 that only doctors tend to select interactive virtual objects regarding toothpaste coupons, a 

2 1 group assignment rule can be created and distributed that dictates that if a reception site 30 has 

22 selected any interactive virtual objects for toothpaste coupons, then the profession of the 

23 subscriber can be set to "doctor." Group assignment rules can be created that then assign, for 

24 each group in a target category, the specific characteristics of a subscriber that are necessary 

25 for a reception site 30 to be assigned to each group. For target category X of Table A, as an 

26 example, group assignment rules can be created and distributed that specifically assign male 

27 doctors and male lawyers to Group B, female doctors and female lawyers to Group C, male 
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1 construction workers to Group D, female construction workers to Group E ? teachers and 

2 college professors to Group F, and anyone else to Group A. Table B below presents an 

3 example of the group assignment rules that would result in this assignment of reception sites to 

4 groups. 

5 As can be seen from this example, group assignment rules can be created to flexibly 

6 manage the subscriber databases within reception sites remotely, while maintaining the privacy 

7 and security of the actual data at the reception site 30. For example, a group assignment rule 

8 may be based on how many times a certain virtual object 38 or several related virtual objects 

9 have been viewed at the reception site 30, allowing for the next in the sequence of virtual 

10 objects to be displayed at the reception site 30 once a requisite number of viewings of 

1 1 subsequent virtual objects have been achieved. 



12 Table B - Example Group Assignment Rules 



13 


Condition (IF) 


Action (THEN) 


Otherwise (ELSE) 


14 


If "Sex field in subscriber 


Then assign reception site to 


Else, go to the next rule 


15 


database" = "Male" AND 


Group B for Target Category 3 




16 


"profession field in subscriber 






17 


database" = "Doctor" OR 






18 


"Lawyer 






19 


If "Sex field in subscriber 


Then assign reception site to 


Else, go to the next rule 


20 


database" = "Female" AND 


Group C for Target Category 3 




21 


"profession field in subscriber 






22 


database" = Doctor" OR 






23 


"Lawyer" 






24 


If "Sex field in subscriber 


Then assign reception site to 


Else, go to the next rule 


25 


database" = "Male" AND 


Group D for Target Category 3 




26 


"profession field in subscriber 






27 


database" = "construction 






28 


worker" 






29 


If "Sex field in subscriber 


Then assign reception site to 


Else, go to the next rule 


30 


database" = "Female" AND 


Group E for Target Category 3 




31 


"profession field in subscriber 






32 


database" = "construction 






33 


worker" 
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1 


Condition (IF) 


Action (THEN) 


Otherwise (ELSE) 


2 


"If "profession field in 


Then assign reception site to 


Else, go to the next rule 


3 


subscriber database" = 


Group F for Target Category 3 




4 


"teacher" OR "college 






5 


professor" 










Then assign reception site to 
Group A for Target Category 3 


End 



6 Figure 1 3 shows an embodiment of the virtual object targeting system 1 220 in more 

7 detail. A resource management engine 1 305 uses information from a metadata extractor 

8 processor 1 200 and an available virtual object database 1 265 (see Figure 9) to determine the 

9 number of virtual objects to be assigned to a given virtual object location 37. A virtual object 

1 0 placement engine 1 307 decides which virtual objects to place in virtual object locations in the 

1 1 content 36. A group assignment engine 1309 determines which groups will view specific virtual 

12 objects. The virtual object placement engine 1307 receives information from the resource 

1 3 management engine 1 305 related to the number of virtual objects available, how many virtual 

14 objects are to be provided for a given virtual object location 37, and the actual type of virtual 

15 objects available. 

1 6 The resource management engine 1 305 functions to divide available delivery bandwidth 

1 7 among multiple virtual objects for a given virtual object location 37 in the content 36. Because 

18 there may be a limited amount ofresourceson the delivery network 11 to deliver virtual objects 

1 9 with the content 36, the resource management engine 1 305 may assign the available bandwidth 

20 optimally for the virtual objects associated with the individual virtual object locations within the 

2 1 content 36 being delivered over the communication channels. Some virtual object locations 

22 may be assigned multiple virtual objects, each targeted to a different group or groups, whereas 

23 other virtual object locations may be assigned only a single virtual object 38. 

24 Referring to Table A, four group numbers (i.e. , 1 -4) are shown for the category of 

25 targeted virtual objects, ADL For a particular virtual object location 37 in the content 36, the 
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1 four groups can be divided into two, one for each available virtual object 38 of two total, with 

2 groups 1 and 2 receiving virtual object A and groups 3 and 4 receiving virtual object B , as 

3 shown for virtual object location 1 . A retrieval plan for this later example is shown in Table C. 



4 Table C - Retrieval Plan 



Virtual Object 
Location 


Target 
Category 


Virtual Object To 
Retrieve 


Groups Assigned to 
Specific Virtual 
Object 


Group Mask 
Assignment 


Virtual Object Location 1 


ADI 


virtual UDject a 


1> 2 


1 1000000000 






Virtual Object B 


3,4 


00110000000 


Virtual Object Location 2 


HH Income 


Virtual Object A 


1,2,3 


11100000000 






Virtual Object B 


4 


00010000000 


Virtual Object Location 3 


Category x 


Virtual Object A 


U 


11000000000 






Virtual Object B 


3 


00100000000 






Virtual Object C 


4 


00010000000 






Virtual Object D 


5 


00001000000 






Virtual Object E 


6 


00000100000 


Virtual Object Location 4 


All 


Virtual Object A 


All 


11111111111 



1 1 After determining how many virtual objects will be needed for each virtual object 

12 location 37 within the content 36, the resource management engine 1305 may also account for 

13 the type of available targeted virtual objects for display and the variety of subscribers 

1 4 (according to group assignment numbers) who may be viewing the content 36. An advertiser 

15 or content provider may provide this information when forwarding virtual objects for insertion. 
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1 In an embodiment, the virtual object placement engine 1307 determines which specific virtual 

2 objects are to be placed in each available virtual object location 37 within the content 36. The 

3 virtual object placement engine 1 307 first receives the list of selected available virtual objects 

4 from the metadata extractor processor 1200 (Figure 9). In cooperation with the resource 

5 management engine 1 305, the virtual object placement engine 1 307 then determines which of 

6 the available virtual objects should be placed in each virtual object location 37 within the 

7 content 36. For example, if the preferred target category for virtual object location 1 is ADI, 

8 the virtual object placement engine 1 307 will select one or more targeted virtual objects 

9 determined by the metadata extractor processor 1 200 to place in that virtual object location 

10 37 . If the demographic or other data assembled by the configuration set-up system 1 205 

1 1 indicates that more than one targeted virtual object 38 should be placed, depending on the 

1 2 ADI, then the virtual object placement engine 1 307 will select the appropriate number of 

1 3 targeted virtual objects, and will assign each targeted virtual object 38 to the specific virtual 

14 object location 37. The operation ofthe virtual object placement engine 1307 to assign the 

15 targeted virtual objects will be described in more detail later. 

16 In an embodiment, the group assignment engine 1 309 receives inputs from the resource 

1 7 management engine 1 305 and the virtual object placement engine 1 307 and then determines 

1 8 which target category groups will view specific targeted virtual objects. Thus, for each virtual 

1 9 object location 37, the group assignment engine 1 309 assigns groups of reception sites to each 

20 ofthe virtual objects. The reception sites may be assigned based on their placement within a 

2 1 group (i.e. , based on their group assignment number) as defined by the group assignment rules 

22 received from the operations center 10. In table C, the assignments are shown based on the 

23 group assignment numbers. As also shown in Table C, the group addressing for a virtual object 

24 location 37 may be based on a single category of targeting. This may avoid a conflict regarding 

25 which virtual object 38 a reception site 30 may retrieve. 

26 The group assignment engine 1 309 provides an output to the retrieval plan generator 
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1 1275. The output indicates which group assignment numbers (i.e., which groups of reception 

2 sites) are assigned to a virtual object 38 for a given virtual object location 37 in the content 36. 

3 The retrieval plan generator 1 275 then generates a bit word, or group mask assignment, that 

4 is used to assign the groups to virtual objects. Once generated, the retrieval plan is provided 

5 to the delivery processor 1 300 for distribution along with the content 36 and the actual virtual 

6 objects to reception sites by object delivery center 15. 

7 In an embodiment, the virtual object targeting system 1 220 provides a virtual object 

8 generation request command 1261 to the content and virtual object packager 1260. The 

9 virtual objects generation request command 1 261 specifies which particular virtual objects are 

10 to be displayed in a particular virtual object location 37, and the actual location of the virtual 

11 objects. The virtaal object 38 is thenretri^^ 1265. 

12 The virtual objects, along with the retrieval plan, and content 36 and associated metadata 

1 3 packets are provided to the delivery processor 1 300 for delivery to the appropriate reception 

14 sites. 

1 5 When a reception site 30 receives the content 36 that contains targeted virtual objects, 

1 6 software instructions operating on the reception site 30 analyze the contents of the retrieval 

17 plan. Then, based on the groups assigned for each virtual object 38, the reception site 30 

1 8 retrieves those virtual objects that match its own group assignments for the target category 

1 9 being used for the virtual object location 37. The reception site 30 then associates those virtual 

20 objects retrieved with the appropriate virtual object location 37 where the virtual object 38 will 

21 be placed, so that when the content 36 is viewed, the virtual object 38 assigned to that virtual 

22 object location 37 is displayed. 

23 An embodiment of the process for assigning targeted virtual objects using the virtual 

24 object placement engine 1307 is presented in Figure 14. The process begins with block 2360. 

25 In block 2362, the reception sites are assigned to groups based on distributed group 

26 assignment rules. Inblock2364, the virtual object placement engine 1307 ties or relates virtual 
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1 object locations in content 36 to the groups. In block 2366, the virtual object placement engine 

2 1307 ties or relates virtual objects to groups. In block 2368, the virtual object placement 

3 engine 1 307 determines how many virtual objects to assign to a virtual object location 37. In 

4 block 2370, the virtual object placement engine 1 307 determines which target category to use 

5 for one or more virtual object locations 37. In block 2372, the virtual object placement engine 

6 1307 determines specific virtual objects to be placed in the virtual object locations 37. In 

7 block 2374, the virtual object placement engine 1307 determines which groups to assign to the 

8 virtual objects 38 for the selected virtual object locations 37. The process ends with block 

9 2376. 

10 As discussed above, virtual object targeting uses target categories and groups within 

1 1 each target category to tie or relate three entities together: 1 ) the reception site 30; 2) virtual 

1 2 objects; and 3) virtual object locations in content 36. In one embodiment of block 2362 in 

13 Figure 14, the reception sites are assigned to groups for each target category by the 

1 4 configuration set-up system 1 205 using the distribution of group assignment rules on which each 

15 reception site 30 assigns itself to a group for each target category. 

16 In one embodiment of block 2364 inFigure 14, virtual object locations in content 36 

17 are tied or related to groups as described below. For each virtual object location 37, a group 

1 8 breakdown percentage can be defined for each group that represents the likely compatibility 

19 of the content 36 surrounding that virtual object location 37 with each group. Breakdown 

20 percentages for each virtual object location 37 are defined within the virtual object selector 200 

2 1 (see Figure 8) and passed to the TVOMS 300. Table D shows a sample breakdown of these 

22 group breakdown percentages for five example virtual object locations for three example target 

23 categories. 

24 The group breakdown percentage data may be derived from information resident in the 

25 viewing population information database 1 2 1 0 or a number of sources including surveys, and 

26 program ratings services such as Nielsen or Arbitron, for example. In this example, the group 
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1 assignment numbers are the same as those presented in Table A. Thus, target categories 1 and 

2 2 each have four groups associated with them, and target category 3 has six groups associated 

3 with it. For virtual object location 1 , the target category 1 refers to ADI and under group 1 , 

4 a group breakdown percentage of 25 percent is assigned for group 1 from the target category 

5 ADI since 25 percent of the subscribers reside in the Seattle, WA ADI. The group breakdown 

6 percentages for each target category for each virtual object location 37 may sum to 100 

7 percent. 

8 In an embodiment of the subroutine represented by block 2366 of Figure 14, virtual 

9 objects may be ranked according to their potential revenue generation for each group within 

1 0 one and up to all possible target categories, again using percentages . This information may be 

1 1 provided by an advertiser, programmer, or content provider responsible for the virtual objects 

1 2 and may reside in the available virtual objects database 1 265 . Information resident in the 

13 viewing population information database 1210 may be used to assign virtual object ranking 

14 percentages. Table E shows a sample assignment of virtual object ranking percentages for 

1 5 eight sample virtual objects using the same target categories and group numbers as in Table D. 

1 6 Not all virtual objects may be assigned to groups for a target category if an advertiser or 

1 7 programmer does not wish its virtual objects to be targeted in the manner required by that 

1 8 target category. For example, an advertiser or programmer may want the same virtual object 

19 to be displayed at all reception sites 30, regardless of subscriber group information or 

20 characteristics. 



2 1 Table D - Virtual Object Location Group Breakdown Percentages 



22 
23 
24 


Virtual 
object 
location 


Target 
Category 


Group 
1 


Group 
2 


Group 
3 


Group 
4 


Group 
5 


Group 
6 


25 


Virtual 


1 


25 


25 


25 


25 


N/A 


N/A 


26 


object 
















27 


location 1 
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Virtual 
object 
location 


Target 
Category 


Group 
1 


Group 
2 


Group 
3 


Group 
4 


Group 
5 


Group 
6 


1 


it 


2 


30 


10 


20 


40 


N/A 


N/A 


2 


a 


3 


10 


20 


30 


40 


N/A 


N/A 


3 
4 
5 


Virtual 
object 
location 2 


1 


10 


20 


30 


40 


N/A 


N/A 


6 


a 


2 


25 


25 


25 


25 


N/A 


N/A 


7 


a 


3 


10 


15 


25 


25 


15 


10 


8 
9 
10 


Virtual 
object 
location 3 


1 


40 


30 


20 


10 


N/A 


N/A 


11 


a 


2 


80 


10 


5 


5 


N/A 


N/A 


12 


a 


3 


25 


25 


10 


10 


15 


25 


13 
14 
15 


Virtual 
object 
location 4 


1 


50 


0 


50 


0 


N/A 


N/A 


16 


« 


2 


0 


40 


40 


20 


N/A 


N/A 


17 




3 


10 


10 


25 


25 


15 


15 


18 
19 
20 


Virtual 
object 
location 5 


1 


20 


30 


30 


20 


N/A 


N/A 


21 


a 


2 


30 


30 


10 


30 


10 


10 


22 


a 


3 


10 


30 


10 


30 


10 


10 



23 Referring to Table E, the data indicates that for virtual object 1, and target category 1 

24 (ADI), the advertiser believes that virtual object 1 is appropriate for the subscribers in groups 

25 1 and 2 and is not appropriate for the subscribers in groups 3 and 4. The advertiser also 
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1 believes that virtual object 1 is equally appropriate for both the group 1 and the group 2 

2 subscribers. However, if the group 1 subscribers are determined to be more likely to respond 

3 to virtual object 1 than the group 2 subscribers, then group 1 could be given a higher 

4 percentage than group 2. Table E also shows that virtual object 1 is not applicable to groups 

5 5 and 6 because only four groups are defined for the target category ADL Thus, all the 

6 reception sites will be grouped into one of groups 1 through 4. 

7 Using this paradigm, virtual objects can be targeted using at least two methods. The 

8 first is a designated multi- virtual obj ect campaign where specific unique sets of groups are 

9 assigned for each virtual object 38 of the campaign. In the second method, each virtual object 

10 38 provided by an advertiser is independently associated with groups. Virtual objects from 

1 1 several different advertisers are then used together to optimize use of virtual object locations. 



12 Table E - Virtual Object Ranking Percentages 



13 


Virtual Object 


Target 


Group 


Group 


Group 


Group 


Group 


Group 






Category 


1 


2 


3 


4 


5 


6 


14 


Virtual object 1 


1 


50 


50 


0 


0 


N/A 


N/A 


15 


ti 


2 


30 


10 


20 


40 


N/A 


N/A 


16 




3 


0 


0 


0 


0 


0 


0 


17 


Virtual object 2 


1 


0 


0 


50 


50 


N/A 


N/A 


18 




2 


0 


0 


0 


0 


N/A 


N/A 


19 




3 


0 


0 


0 


0 


0 


0 


20 


Virtual obiect3 


1 


0 


0 


0 


0 


N/A 


N/A 


21 




2 


25 


25 


25 


25 


N/A 


N/A 


22 




3 


0 


0 


0 


0 


0 


0 


23 


Virtual object 4 


1 


50 


0 


50 


0 


N/A 


N/A 


24 




2 


0 


40 


40 


20 


N/A 


N/A 


25 




3 


10 


30 


10 


30 


10 


10 


26 


Virtual object 5 


1 


40 


20 


20 


40 


N/A 


N/A 


27 


<t 


2 


10 


30 


30 


30 


N/A 


N/A 


28 


it 


3 


30 


30 


30 


5 


5 


0 


29 


Virtual object 6 


1 


0 


0 


0 


0 


N/A 


N/A 


30 




2 


0 


0 


0 


0 


N/A 


N/A 


31 


it 


3 


10 


10 


10 


10 


30 


30 


32 


Virtual object 7 


1 


20 


40 


40 


20 


N/A 


N/A 


33 


« 


2 


25 


25 


25 


25 


N/A 


N/A 


34 




3 


0 


30 


20 


30 


0 


20 


35 


Virtual object 8 


1 


30 


40 


0 


30 


N/A 


N/A 


36 


a 


2 


30 


30 


10 


30 


N/A 


N/A 


37 


u 


3 


20 


0 


20 


20 


20 


20 
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1 As depicted in Figure 14, blocks 2368, 2370, 2372, and 2374, the virtual object 

2 placement engine 1 307 determines: 1) how many virtual objects are assigned to which virtual 

3 object location; 2) which target category is used for which virtual object location; 3) which 

4 virtual objects to place in each virtual objects location; and 4) which groups are assigned to 

5 which virtual objects, respectively. To limit the need for excessive distribution bandwidth to 

6 distribute virtual objects to reception sites, the algorithm in the virtual object placement engine 

7 1 307 that assigns targeted virtual objects to the virtual objects assumes that there is a total 

8 number of virtual objects available [TOTAL„VIRTUAL OBJECTS] for a segment of content 

9 36 (across all virtual object locations), and assumes that no more than some maximum number 

1 0 of the virtual objects can be or are desired to be assigned to a given virtual object location 37. 

1 1 This amount is denoted as [MAX_VIRTUAL OBJECTS] . 

1 2 Figure 1 5 presents an embodiment of a process used by the virtual object placement 

13 engine 1 307 to execute the functions listed in blocks 2368, 2370, 2372, and 2374 depicted 

14 inFigure 14. The process begins with the start ellipse, 2318. In block 2320, the virtual object 

1 5 placement engine 1 307 determines the virtual object 38 best suited for each virtual object 

16 location 37 for all target categories. In block 2322, the virtual object placement engine 1307 

1 7 determines the best virtual object/target category combination for each virtual object location 

18 37. In block 2324, the virtual object placement engine 1 307 compares virtual object/target 

19 category combinations for all virtual object locations. In block 2326, the virtual object 

20 placement engine 1 307, for a virtual object location 37 and target category, determines the best 

2 1 virtual objects to associate with the virtual object location 37. In block 2328, the virtual object 

22 placement engine 1307 repeats block 2326 for each target category. In block 23 30, the virtual 

23 objectplacementengine 1 307 determines the target category that yields the optimum placement 

24 of virtual objects for a virtual object location 37. In block 2332, the virtual object placement 

25 engine 1307 repeats blocks 2326, 2328, and 2330 for all virtual object locations. Inblock 

26 2334, the virtual object placement engine 1 307 determines the best combination of multiple 
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1 virtual objects for each virtual object location 37. In block 2336, for the remaining virtual 

2 object locations, the virtual obj ect placement engine 1 307 assigns the best matching virtual 

3 object 38. The process ends with block 2338. 

4 A further embodiment of a virtual objects targeting algorithm presented in Figure 15 

5 will be described with reference to the example values shown in Tables A-E. Various other 

6 prioritizing or ranking schemes may be used as described later. 

7 Step 1: In block 2320 in Figure 15, the virtual object placement engine 1307,fora 

8 virtual object location 37, determines the virtual objects with the highest overall ranking if that 

9 virtual object 38 were the only virtual object 38 to be placed in a virtual object location 37 in 

1 0 the content 36. This step compares the data in Tables D and E. Figure 1 6 and the description 

1 1 that follows below present a more detailed embodiment of several of the blocks presented in 

12 Figure 15. In step la, as an embodiment of block 2421 in Figure 16, the virtual object 

13 placement engine 1307 selects the first virtual object location 37 and as an embodiment of 

14 block2421 inFigure 16, selects the first virtual object 38 to be analyzed. As Step lb, for that 

1 5 virtual object selected in Step 1 a, the virtual object placement engine 1 307 selects the first 

16 category, as an embodiment ofblock 2423 in Figure 16. Then, the virtual obj ect placement 

17 engine 1 307 multiplies the virtual object's Group Ranking Percentage by the virtual object 

1 8 location's Group Breakdown Percentage for each group as an embodiment ofblock 2424 in 

19 Figure 16 and sums the result, as an embodiment ofblock 2425 in Figure 16. AsStep lc,the 

20 virtual object placement engine 1307 repeats Step lb for the next target category, as an 

21 embodiment ofblock 2426 in Figure 16. AsStep Id, the virtual object placement engine 1307 

22 repeats steps lb and lc for each virtual object 38, as an embodiment ofblock 2427 in Figure 

23 16. As Step le, for the virtual object location 37 under consideration, the virtual object 

24 placement engine 1307 selects the virtual object/target category that yields the highest summed 

25 value, as an embodiment of block 2428 in Figure 16. Then, for Step If, the virtual object 
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11 
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14 
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16 
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18 
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placement engine 1 307 repeats Steps lb-le for all virtual object locations, as an embodiment 

of block 2429 in Figure 16. 

For example, using virtual object location 1, virtual object 1: 

target category 1: 50*25 + 50*25 + 0*25 + 0*25 = 25% 

target category 2: 30*30 + 10*10 + 20*20 + 40*40= 30% 

target category 3: 0*10 + 0*10 + 0*20 + 0*20 0*20 + 0*20 = 0% 

The cross-multiplied result then shows a measure of effectiveness for each virtual 

object 38 if displayed in the corresponding virtual object location 37. Table F below presents 

the results of Step 1 above for virtual object location 1. 

Table F 



19 



Virtual 
object 
location/ 
Virtual 
object 


Target 
Category 


Group 
1 


Group 
2 


Group 
3 


Group 
4 


Group 
5 


Group 
6 


Sum- 
mation 


1/1 


1 


12.5 


12.5 


0 


0 


0 


0 


25 




2 


9 


1 


4 


16 


0 


0 


30 




3 


0 


0 


0 


0 


0 


0 


0 


1/2 


1 


0 


0 


12.5 


12.5 


0 


0 


25 




2 


0 


0 


0 


0 


0 


0 


0 




3 


0 


0 


0 


0 


0 


0 


0 


1/3 


1 


0 


0 


0 


0 


0 


0 


0 




2 


7.5 


2.5 


5 


10 


0 


0 


25 




3 


0 


0 


0 


0 


0 


0 


0 


1/4 


1 


12.5 


0 


12.5 


0 


0 


0 


25 




2 


0 


4 


8 


8 


0 


0 


20 
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Virtual 

0 DJ C CI 

location/ 
Virtual 
object 


Target 
Category 


Group 
1 


Group 
2 


Group 
3 


Group 
4 


Group 
5 


Group 
6 


Sum- 
mation 




3 


1 


3 


2 


6 


2 


2 


16 


1/5 


1 


10 


5 


5 


5 


0 


0 


25 




2 


3 


3 


6 


12 


0 


0 


24 




3 


3 


3 


6 


1 


1 


0 


14 


1/6 


1 


0 


0 


0 


0 


0 


0 


0 




2 


0 


0 


0 


0 


0 


0 


0 




3 


1 


1 


2 


2 


6 


6 


18 


1/7 


1 


5 


5 


10 


5 


0 


0 


25 




2 


7.5 


2.5 


5 


10 


0 


0 


25 




3 


0 


3 


4 


6 


0 


4 


17 


1/8 


1 


7.5 


10 


0 


7.5 


0 


0 


25 




2 


9 


3 


2 


12 


0 


0 


26 




3 


2 


0 


4 


4 


4 


4 


18 



5 Step 2: Returning to Figure 15,foreachvirtualobjectlocation37,the virtual object 

6 placement engine 1307, in block 2322, determines the virtual object/target category 

7 combination that results in the highest overall ranking. In one embodiment the virtual object 

8 placement engine 1307, liststhe virtual object locations, the overall ranking, the corresponding 

9 virtual object 38, and the corresponding target category. In case of a tie, the virtual object 

10 placement engine 1307 selectsany virtual object 38 with the overall highest ranking. TableG 

1 1 shows the results. Thus, from Table G, virtual object 4, a virtual object 38 displayed within 
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1 virtual object location 4 yields a measure of effectiveness of 50 (highest) and virtual object 8 

2 along within virtual object location 5 yields a measure of effectiveness of 28. 



3 Table G 



4 


Virtual Object Location 


Highest Overall 
Ranking 


Corresponding 
Virtual Object 


Corresponding 
Target Category 


5 


Virtual object location 1 


30 


Virtual Object 1 


2 


6 


Virtual object location 2 


35 


Virtual Object 2 


1 


7 


Virtual object location 3 


35 


Virtual Object 1 


1 


8 


Virtual object location 4 


50 


Virtual Object 4 


1 


9 


Virtual object location 5 


28 


Virtual Object 8 


2 



10 Step 3: In one embodiment ofblock 2324 in Figure 15, the virtual object placement 

1 1 engine 1 307 orders the resulting list of virtual object locations from Step 2 from lowest overall 

1 2 ranking to highest overall ranking to compare virtual object/target category combinations for 

13 virtual object locations. Table H shows the results. 



14 Table H 



15 


Virtual Object Location 


Overall 
Ranking 


Corresponding 
Virtual Object 


Corresponding 
Target Category 


16 


Virtual object location 5 


28 


Virtual Object 8 


2 


17 


Virtual object location 1 


30 


Virtual Object 1 


2 


18 


Virtual object location 2 


35 


Virtual Object 2 


1 


19 


Virtual object location 3 


35 


Virtual Object 1 


1 


20 


Virtual object location 4 


50 


Virtual Object 4 


1 
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1 Step 4: In one embodiment of block 2326 in Figure 15, the virtual object placement 

2 engine 1307 uses the process shown in Figure 17 to determine the best virtual objects to 

3 associate with a virtual object location 37. The process begins with ellipse 2440. In block 

4 2441 inFigure 17, the virtual object placement engine 1307 selects the virtual object location 

5 37 from Step 3 resulting in the lowest overall ranking. As Step 4a, for the selected virtual 

6 object location 37, the virtual object placement engine 1 307 selects the first target category, 

7 as an embodiment ofblock2442 inFigure 17. As Step 4b, the virtual object placement engine 

8 1 307 assembles a table showing the product of each virtual object Group Ranking Percentage 

9 and virtual object location Group Breakdown Percentage combination. Table Ibelow provides 
10 an example for virtual object location 5 and target category L 



11 Table I 



12 
13 
14 


Virtual Object 
Location / Virtual 
Object 


Target 
Category 


Group 
1 


Group 
2 


Group 
3 


Group 
4 


Sum- 
mation 


15 


5/1 


1 


10 


15 


0 


0 


25 


16 


5/2 


1 


0 


0 


15 


10 


25 


17 


5/3 


1 


0 


0 


0 


0 


0 


18 


5/4 


1 


10 


0 


15 


0 


25 


19 


5/5 


1 


8 


6 


6 


4 


24 


20 


5/6 


1 


0 


0 


0 


0 


0 


21 


5/7 


1 


4 


6 


12 


4 


26 


22 


5/8 


1 


6 


12 


0 


6 


24 



-42- 



Docket 5220/PTO Filings/Spec.wpd 

1 As Step 4c, as an embodiment of block 2443 in Figure 17, the virtual object 

2 placement engine 1 307 finds the product that is the highest. Li case of a tie, the virtual object 

3 placement engine 1 307 selects the product that corresponds to the highest summation value for 

4 that virtual object location / virtual object combination. In case a tie still persists, the virtual 

5 object placement engine 1307 selects any of thecells with an equivalent value. TableJbelow 

6 shows the previous example continued where group 2 for virtual object location / virtual object 

7 combination 5/1 is selected. 



8 Table J 



9 
1U 
11 


Virtual Object 
Location / Virtual 
Object 


Target 
Category 


Group 
1 


Group 
2 


Group 
3 


Group 
4 


Sum- 


12 


5/1 




10 


*15* 


0 


0 


25 


13 


5/2 




0 


0 


15 


10 


25 


14 


5/3 




0 


0 


0 


0 


0 


15 


5/4 




10 


0 


15 


0 


25 


16 


5/5 




8 


6 


6 


4 


24 


17 


5/6 




0 


0 


0 


0 


0 


18 


5/7 




4 


6 


12 


4 


26 


19 


5/8 




6 


12 


0 


6 


24 



20 Step5: As an embodiment of block 2444 in Figure 17, the virtual object placement 

2 1 engine 1 307 finds the product that is next highest (or the same value as in Step 4), but that is 

22 associated with a group not yet selected. Again, in case of a tie, the virtual object placement 

23 engine 1307 selects the product that corresponds to the highest summation value for that virtual 
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1 object location / virtual object combination. In case a tie still persists, the virtual object 

2 placement engine 1307 selects any of the cells with an equivalent value. Table Kbelow shows 

3 the previous example continued. 

4 Table K 



5 
6 
7 


Virtual Object 
Location /Virtual 
Object 


Target 
Category 


Group 1 


Group 2 


Group 3 


Group 4 


8 


5/1 




*10* 


*15* 


0 


0 


9 


5/2 




0 


0 


*15* 


*10* 


10 


5/3 




0 


0 


0 


0 


11 


5/4 




10 


0 


15 


0 


12 


5/5 




8 


6 


6 


4 


13 


5/6 




0 


0 


0 


0 


14 


5/7 




4 


6 


12 


4 


15 


5/8 




6 


12 


0 


6 



16 Step6: As an embodiment of block 2446 in Figure 17, the virtual object placement 

17 engine 1307 repeats Step 5 until a product has been selected for all groups. TableLbelow 

1 8 continues the example. 

19 Table L 



20 
21 
22 


Virtual Object 

Location / 
Virtual Object 


Target 
Category 


Group 1 


Group 2 


Group 3 


Group 4 


23 


5/1 


1 


*10* 


*15* 


0 


0 
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Virtual Object 

Location / 
Virtual Object 


Target 
Category 


Group 1 


Group 2 


Group 3 


Group 4 


1 


5/2 


1 


0 


0 


*15* 


*10* 


2 


5/3 


1 


0 


0 


0 


0 


3 


5/4 


1 


10 


0 


15 


0 


4 


5/5 


1 


8 


6 


6 


4 


5 


5/6 


1 


0 


0 


0 


0 


6 


5/7 


1 


4 


6 


12 


4 


7 


5/8 


1 


6 


12 


0 


6 



8 Step 7: As an embodiment of block 2448 in Figure 17, for all virtual objects with 

9 products cells selected in Step 6, the virtual object placement engine 1307 calculates the 

10 summed products of those selectedcellsforeachvirtual object 38. TableMbelow shows the 

11 results. 



12 Table M 



13 
14 
15 


Virtual Object 

Location / 
Virtual Object 


Target 
Category 


Group 1 


Group 2 


Group 3 


Group 4 


Sum- 
mation 


16 


5/1 


1 


*10* 


*15* 


0 


0 


25 


17 


5/2 


1 


0 


0 


*15* 


*10* 


25 


18 


5/3 


1 


0 


0 


0 


0 


0 


19 


5/4 


1 


10 


0 


15 


0 


0 
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virtual UDjeci 

Location / 
Virtual Object 


Target 
Category 


Group 1 


Group 2 


Group 3 


Group 4 


Sum- 
mation 


1 


5/5 


1 


8 


6 


6 


4 


0 


2 


5/6 


1 


0 


0 


0 


0 


0 


3 


5/7 


1 


4 


6 


12 


4 


0 


4 


5/8 


1 


6 


12 


0 


6 


0 



5 Step8: As an embodiment ofblock 2450 in Figure 17, the virtual object placement 

6 engine 1 307 orders the virtual objects in Step 7 from highest summed value to lowest. In case 

7 of equal summed values, the virtual object placement engine 1 307 arbitrarily orders those 

8 virtual objects with the same summed value. Table N presents the example results. 



9 Table N 



10 

11 

12 


Virtual Object 

Location / 
Virtual Object 


Target 
Category 


Group 1 


Group 2 


Group 3 


Group 
4 


Sum- 
mation 


13 


5/1 


1 


10 


15 


0 


0 


25 


14 


5/2 


1 


1 


0 


15 


10 


25 



15 Step 9: As Step 9a, if the number of virtual objects selected in Step 8 exceeds 

16 [MAX_VIRTUAL OBJECTS], the virtual object placement engine 1307 selects the first 

17 [MAX_VIRTUAL OBJECTS] virtual objects with the summed value as an embodiment of 

18 block 2452 in Figure 17. Forexample,ifitisdesiredtoassignatmosttwovirtualobjectsto 

19 a virtual object location 37, the virtual object placement engine 1307 selects the two virtual 
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1 objects with the highest virtual object Group Ranking Percentage and virtual object location 

2 Group Breakdown Percentage products. Next, as Step 9b, for the unselected virtual obj ects, 

3 the virtual object placement engine 1 307 determines those groups that were associated with 

4 these omitted virtual objects, as an embodiment of block 2454 in Figure 17. 

5 Step 10: As an embodiment of block 2456 in Figure 17, for the virtual objects 

6 associated with the groups determined in Step 9b, the virtual object placement engine 1 307 

7 selects the product within that group that is the highest for the [MAX_VIRTUAL OBJECT] 

8 selected virtual objects from Step 9a. The virtual object placement engine 1 307 recalculates 

9 the summed products of those selected groups cells for each of the virtual objects. Table O 

10 below provides a new example, assuming [MAX_VIRTUAL OBJECTS] = 2; therefore, 

1 1 groups 5 and 6, which are associated with virtual object 6, maybe reallocated to virtual objects 

12 7 & 5, respectively. 

13 Table O 



14 Result before Step 10 is shown below: 



15 
16 
17 
18 
19 


Virtual 
object 
location/ 
Virtual 
object 


Target 
Category 


Group 
1 


Group 
2 


Group 
3 


Group 
4 


Group 
5 


Group 
6 


Sum- 
mation 


20 


5/7 


3 


0 




2 


#9* 


0 


2 


18 


21 


5/5 


3 


*3* 


9 


#3* 


1.5 


0.5 


0 


6 


22 


5/6 


3 


1 


3 


1 


3 


#3* 


! 

#3* 


6 



23 Result after Step 10 is shown below: 
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1 

2 

D 

4 
5 


Virtual 
object 
location / 

Virtual 
object 


Target 
Category 


Group 
1 


Group 
2 


Group 
3 


Group 
4 


Group 
5 


Group 
6 


Sum- 
mation 


6 


5/7 


3 


0 




2 




0 




20 


7 


5/5 


3 


*3* 


9 


*3* 


1.5 


*0.5* 


0 


6.5 


8 


5/6 


3 


1 


3 


1 


3 


3 


3 


0 



9 Step 11: As an embodiment of block 2458 in Figure 17, the virtual object placement 

1 0 engine 1 307 calculates the total summed product value for all virtual objects selected in Step 

11 10. From Table P, this value is 26.5. The resultant groups selected for each virtual object 38 

1 2 will serve as the group assignments if this virtual object location / target category ultimately 

13 results in the best match, as determined in the remaining steps of the algorithm. 



14 Table P 



15 
16 
17 
18 
19 


Virtual 
object 
location/ 
Virtual 
object 


Target 
Category 


Group 
1 


Group 
2 


Group 
3 


Group 
4 


Group 
5 


Group 
6 


Sum- 
mation 


20 


5/7 


3 


0 


#9* 


2 


$9$ 


0 


*2* 


20 


21 


5/5 


3 


#3* 


9 


*3* 


1.5 


0.5 


0 


6.5 


22 




rotal summed product values 


26.5 



23 Step 12: The virtual object placement engine 1307 repeats steps 4- 11 above for the 

24 same selected virtual object location 37 of Step 4 using the remaining target categories, as an 

25 embodiment of block 2328 in Figure 1 5. The Table Q example below provides the output 

26 results for each of the three example target categories. 
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1 Table Q 



2 
3 
4 
5 
6 


Virtual 
object 
location / 

Virtual 
object 


Target 
Category 


Group 
1 


Group 
2 


Group 
3 


Group 
4 


Sum- 
mation 


7 


5/1 


1 


*10* 


*15* 


0 


0 


25 


8 


5/2 


1 


0 


0 


*15* 


*10* 


25 


9 


Total summed product values 


50 



10 

11 

12 
13 
14 


Virtual 
object 
location / 

Virtual 
object 


Target 
Category 


Group 
1 


Group 
2 


Group 
3 


Group 
4 


Group 
5 


Group 
6 


Sum- 
mation 


15 


5/1 


2 


#9* 


3 


2 


*12* 


0 


0 


21 


16 


5/4 


2 


0 


*12* 


#4* 


6 


0 


0 


16 


17 


Total summed product values 


37 



18 


Virtual 


Target 


Group 


Group 


Group 


Group 


Group 


Group 


Sum- 


19 


object 


Category 


1 


2 


3 


4 


5 


6 


mation 


20 


location 


















21 


/Virtual 


















22 


object 


















23 


5/7 


3 


0 


#9* 


2 


#9* 


0 


*2* 


20 
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Virtual 
object 
location 
/Virtual 
object 


Target 
Category 


Group 
1 


Group 
2 


Group 
3 


Group 
4 


Group 
5 


Group 
6 


Sum- 
mation 


5/5 


3 




9 




1.5 




0 


6.5 


Total summed product values 


26.5 



3 Step 13: As an embodiment ofblock 2330 in Figure 15, the virtual object placement 

4 engine 1 307 selects the target category that yields the highest total summed product amount. 

5 The virtual object placement engine 1 307 assigns this as the Maximum Rank for that virtual 

6 object location 37. In the case above, the virtual object placement engine 1 307 would assign 

7 target category 1, with a value of 50 that is selected. 

8 Step 14: As an embodiment ofblock 2332 in Figure 15, the virtual object placement 

9 engine 1 307 repeats Steps 4- 1 3 for the virtual obj ect location 37 selected in Step 4 with the 

10 next lowest overall ranking, computing the Maximum Rank for each virtual object 

11 location 37. 

12 Step 15: As an embodiment ofblock 2334 in Figure 15, the virtual object placement 

13 engine 1307 uses the available [MAX__VIRTUAL OBJECTS] virtual objects for the virtual 

14 objectlocationsuptothemaximumnumberof[TOTAL_VIRTO 

1 5 largest Maximum Rank. The virtual object placement engine 1307 makes use of the relevant 

1 6 target category determined in Step 13, with virtual objects as determined in Step 10, with 

17 group assignments as determined in Step 11. 

18 Step 16: As an embodiment ofblock 23 36 in Figure 15, for all other virtual object 

19 locations, the virtual object placement engine 1307 assigns the single virtual objects that yielded 

20 the highest Overall Ranking as determined in Step 2. 



-50- 



Docket 5220/PTO Filings/Spec.wpd 

1 The above algorithm performed by the virtual object placement engine 1307ismeant 

2 to be illustrative and not limiting. Other algorithms are possible for assigning targeted virtual 

3 objects to groups of reception sites or to individual reception sites. Other targeted virtual 

4 object routines can also be used by the virtual object placement engine 1307. 

5 Figure 18 depicts the object delivery center 15. The object delivery center 15 

6 receives content 36, virtual objects, retrieval plans, group assignment rules, and other 

7 information from the operations center 10 that is to be transmitted to reception sites. The 

8 communication processor 16 in the object delivery center 15 may determine the delivery 

9 network and communications methods appropriate for each item to be delivered, may combine 

1 0 items to be delivered to common destinations, may format the items for delivery, and provide 

1 1 the formatted items to the processing router 17. The processing router 17 may then route each 

1 2 item to the appropriate modular connector 700, for example modular connector 700', modular 

1 3 connector 700", or modular connector 700 w , depending on the required delivery network 1 1 

14 and communication method. 

15 Anumberofembodimentsofdehverynetworks 11, 12, and 14 are presented below. 

1 6 The embodiments presented below may use the object delivery center 1 5 , which inserts the 

17 virtual objects into the signal for delivery over the delivery network 11 or 12. The 

1 8 embodiments presented below use a modular connector 700 in the reception site 30, that 

1 9 receives the delivered signal with virtual objects, extracts the virtual objects, and provides the 

20 virtual objects to the storage management processor 710. The modular connector 700 

2 1 supports the receive functionality for each unique delivery network communication method 

22 embodiment. 

23 Figure 1 9 presents embodiments associated with the delivery of virtual objects over 

24 a coaxial or fiber cable system 270 1 to a reception site 30. Virtual objects are provided to the 

25 delivery network 1 1 by the object delivery center 15 or directly by the operations center 10. 

26 Alternatively, content 36 and virtual objects may be provided to the reception site 30 from the 
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1 object delivery center 15 or from the local insertion center 20 using delivery network 12. The 

2 signal is delivered over the cable system 270 1 delivery network. The signal may provide for 

3 the delivery of virtual objects, content 36 containing virtual object locations, and group 

4 assignment rules and retrieval plans. The signal may also provide for interactive virtual object 

5 requests from the reception site 30 to the interactive object service center 60 using delivery 

6 network 1 4 or the signal may be a means to provide access to the Internet or other public 

7 network through which virtual objects or content 36 are delivered (not shown) . The cable 

8 system 270 1 may be a coaxial cable network, totally fiber network, hybrid fiber coax network, 

9 fiber to the curb network, or any other cable distribution technology. The signal over the cable 

10 systemmay be generated by a cable modem, in which an external cable modem 2702 is used 

1 1 to receive the signal and provide the embedded virtual objects to the modular connector 700 

12 in the reception site 30 for processing. Alternatively, the reception site 30 may contain an 

1 3 internal cable modem 2705, which receives the signal and provides the virtual objects to the 

14 modular connector 700 for processing. 

1 5 In another embodiment, the signal delivered over the cable system is a video signal. 

1 6 In one embodiment, the video signal is an analog video signal. In another embodiment, the 

17 video signal is a digital video signal. The reception site 30 may contain an internal cable 

1 8 receiver/tuner/demodulator 2706 to process the signal, and provide the embedded virtual 

19 objects to the modular connector 700. A set top terminal 2703, or other device capable of 

20 receiving a cable video signal, such as a cable ready TV, or PC with cable tuner (not shown), 

2 1 may process the video signal and deliver the video signal to the connector 700 in the reception 

22 site 30, which extracts the embedded virtual objects. Alternately, the set top terminal 2703, 

23 or other such device, may extract the embedded virtual objects from the video signal and 

24 provide the virtual objects to the modular connector 700 in the reception site 30. 

25 In another embodiment, virtual objects may be embedded within the audio signal, 

26 requiring an appropriate audio-capable modular connector 700 in the reception site 30 to 
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1 extract the virtual objects from the audio signal. In one embodiment, the audio signal is an 

2 analog audio signal. In another embodiment, the audio signal is a digital audio signal. 

3 In yet another embodiment, the signal is a spread spectrum signal containing a digital 

4 data stream, requiring an appropriate spread spectrum receiver and modular connector 700 

5 in the reception site 30 to extract the virtual objects. In this embodiment, the spread spectrum 

6 signal is transmitted in the same bandwidth as the video or audio signal, but below the noise 

7 level. 

8 Figure 20 presents embodiments associated with the delivery of virtual objects over 

9 a wireless broadcast system 2801 toareceptionsite30. Virtual objects are provided to the 

1 0 delivery network 1 1 by the object delivery center 1 5 or directly by the operations center 1 0. 

1 1 Alternatively, content 36 and virtual objects maybe provided to the reception site 30 from the 

12 object delivery center 15 or from the local insertion center 20 using delivery network 12. The 

1 3 signal is delivered over the wireless broadcast system 2801 delivery network. The signal may 

14 provide for the delivery of virtual objects, content 36 containing virtual object locations, and 

1 5 group assignment rules and retrieval plans. The signal may also provide for interactive virtual 

16 object requests from the reception site 30 to the interactive object service center 60 using 

1 7 delivery network 14 or the signal may be a means to provide access to the Internet or other 

18 public network through which virtual objects or content 36 are delivered. The wireless 

1 9 broadcast system may be a microwave multipoint delivery system (MMDS), local multipoint 

20 distribution system (LMDS), Instructional Television Fixed Service (TTFS) system, or any other 

21 wireless data, video, or telephony broadcast system, including point-to-point and 

22 point-to-multipoint microwave broadcast systems like those provided by Teligent, Winstar 

23 digital wireless network, and ATT's wireless system. The signal over the wireless broadcast 

24 system may be generated by a wireless modem, in which an external wireless modem 2802 is 

25 used to receive the signal and provide the embedded virtual objects to the modular connector 

26 700 in the reception site 30 for processing. Alternatively, the reception site 30 may contain an 
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1 internal wireless modem 2805 , which receives the signal and provides the virtual objects to the 

2 modular connector 700 in the reception site 30 for processing. 

3 In another embodiment, the signal delivered over the wireless broadcast system is a 

4 video signal. In one embodiment, the video signal is an analog video signal. In another 

5 embodiment, the video signal is a digital video signal. The reception site 30 may contain an 

6 internal wireless receiver/tuner/demodulator 2806 to process the signal, and provide the 

7 embedded virtual objects to the modular connector 700. A wireless set-top terminal 2803, or 

8 other device capable of receiving a wireless video signal, such as a TV, or PC with a wireless 

9 receiver and tuner, may process the video signal and deliver the video signal to the modular 

10 connector 700 in the reception site 30, which extracts the embedded virtual objects. 

1 1 Alternately, the set top terminal 2803, or other such device, may extract the embedded virtual 

12 objects from the video signal and provide the data to the modular connector 700 in the 

13 reception site 30. 

1 4 In another embodiment, virtual objects may be embedded within the audio signal, 

1 5 requiring an appropriate audio-capable modular connector 700 in the reception site 30 to 

1 6 extract the virtual objects from the audio signal. In one embodiment, the audio signal is an 

17 analog audio signal. In another embodiment, the audio signal is a digital audio signal. 

18 In yet another embodiment, the signal is a spread spectrum signal containing a digital 

1 9 data stream, requiring an appropriate spread spectrum receiver modular connector 700 in the 

20 reception site 30 to extract the virtual objects. In this embodiment, the spread spectrum 

2 1 signal is transmitted in the same bandwidth as the video or audio signal, but below the noise 

22 level. 

23 Figure 2 1 presents embodiments associated with the delivery of virtual objects over 

24 a satellite broadcast system 2901 to areception site 30. Virtual objects are provided to the 

25 delivery network 1 1 by the object delivery center 1 5 or directly by the operations center 1 0. 

26 Alternatively, content 36 and virtual objects may be provided to the reception site 30 from the 
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1 object delivery center 15 or from the local insertion center 20 using delivery network 12. The 

2 signalisdeliveredoverthesatelUtebroadcastsystem2901 delivery network. Thesignalmay 

3 provide for the delivery of virtual objects, content 36 containing virtual object locations, and 

4 group assignment rules and retrieval plans. The signal may also provide for interactive virtual 

5 object requests from the reception site 30 to the interactive object service center 60 using 

6 delivery network 14 or the signal may be a means to provide access to the Internet or other 

7 public network through which virtual objects or content 36 are delivered. The satellite 

8 broadcast system 290 1 can be a direct broadcast system like DirecTV and EchoStar, a direct 

9 to home satellite broadcast system, video network distribution broadcast system, a 
1 0 point-to-point or point-to-multipoint data VS AT system, a digital audio broadcast system like 

O 1 1 WorldSpace, Sirius - formerly CD Radio, or XM, or a mobile data and telephony satellite 

IP 1 2 broadcast system like Iridium, Teledesic, or Globalstar. Alternatively, the satellite broadcast 

m 13 system can be regionalized broadcast services or store and forward communication services 

2 14 hosted on high flying balloons or on airplanes that provide communication repeater services to 

' mi 15 an small geographic region. The signal over the satellite broadcast system may be generated 

D 16 by a satellite data modem, in which an external satellite data receiver 2902 is used to receive 

lj 1 7 the signal and provide the embedded virtual objects to the reception site 30 modular connector 

1 8 700 for processing. Alternatively, the reception site 30 may contain an internal satellite receiver 

u 1 9 2905, which receives the signal and provides the virtual objects to the modular connector 700 

20 in the reception site 30 for processing. 

21 In another embodiment, the signal delivered over the satellite broadcast system is a 

22 video signal. In one embodiment, the video signal is an analog video signal. In another 

23 embodiment, the video signal is a digital video signal. The reception site 30 may contain an 

24 internal satellite video receiver 2906 to process the signal, and provide the embedded virtual 

25 objects to the modular connector 700. A satellite receiver 2903, or other device capable of 

26 receiving a satellite video signal, such as a TV, or PC with satellite receiver, may process the 
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1 video signal and deliver the video signal to the modular connector 700 in the reception site 30, 

2 which extracts the embedded virtual objects. Alternately, the satellite receiver 2903, or other 

3 such device, may extract the embedded virtual objects from the video signal and provide the 

4 data to the modular connector in the reception site 258. 

5 In another embodiment, virtual objects may be embedded within the audio signal, 

6 requiring an appropriate audio-capable modular connector 700 in the reception site 30 to 

7 extract the virtual objects from the audio signal. In one embodiment, the audio signal is an 

8 analog audio signal. In another embodiment, the audio signal is a digital audio signal. 

9 In yet another embodiment, the signal is a spread spectrum signal containing a digital 

1 0 data stream, requiring an appropriate spread spectrum receiver modular connector 700 in the 

1 1 reception site 30 to extract the virtual objects. In this embodiment, the spread spectrum 

1 2 signal is transmitted in the same bandwidth as the video or audio signal, but below the noise 

13 level. 

14 Figure 22 presents embodiments associated with the delivery of virtual objects over 

15 awireddatanetwork3001 to a reception site 30. Virtual objects are provided to the delivery 

16 network 11 by the object delivery center 15 or directly by the operations center 10. 

17 Alternatively, content 36 and virtual objects may be provided to the reception site 30 from the 

1 8 object delivery center 1 5 or from the local insertion center 20 using delivery network 12. The 

19 signalis delivered overthe wired data network 3001 delivery network. The signal may provide 

20 for the delivery of virtual objects, content 36 containing virtual object locations, and group 

21 assignmentrules and retrieval plans. The signal may also provide for interactive virtual object 

22 requests from the reception site 30 to the interactive object service center 60 using delivery 

23 network 14 or the signal may be a means to provide access to the Internet or other public 

24 network through which virtual objects or content 36 are delivered. The wired data network 

25 3001 can be metallic wire or fiber, supporting any of a number of communication standards 

26 including HDSL, ADSL, DSL, ISDN, T 1 , T3, SONET, ATM, X.25, frame relay, Switched 
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1 MultiMegabit Data Service (SMDS), or others. The signal sent over the wired data network 

2 maybe generated by a datamodem or transmission device, in which the appropriate modem, 

3 interface device, or DataTerrninating Equipment (DTE) device is used to receive the signal and 

4 provide the embedded virtual objects to the reception site 30 modular connector 700 for 

5 processing. Embodiments of such receiving devices are shown in Figure 22 as HDSL modem 

6 3002, ADSL modem 3003, DSL modem 3003, ISDN Terminal equipment (TE) device 3005, 

7 Tl Digital service urrit(DSU) 3006, T3DSU3()07,Fta 

8 3008, ATMUNI3009,X.25DTE3010,Framerelay assembler/disassembler (FRAD) 3011, 

9 andSMDS subscriber network interface device (SN1) 3012. Alternatively, the reception site 

10 30 may contain an internal modem or DTE 3013, which receives one or more signal types and 

1 1 provides the received signal with embedded virtual objects to the modular connector 700 in the 

12 reception site 30 for processing. Finally, the reception site 30 may be attached to a wired 

13 LAN using atransceiver. Inthis embodiment, virtual objects maybe delivered over theLAN 

14 at any time. 

1 5 Figure 23 presents embodiments associated with the delivery of virtual objects using 

16 the public switched telephone network (PSTN) 3 10 1 to a reception site 30. Virtual objects 

17 are provided to the delivery network 1 1 by the object delivery center 15 or directly by the 

18 operations center 10. Alternatively, content 36 and virtual objects maybe provided to the 

1 9 reception site 30 from the obj ect delivery center 1 5 or from the local insertion center 20 using 

20 delivery network 12. The signal is delivered over the PSTN 3101 delivery network. The 

2 1 signal may provide for the delivery of virtual objects, content 36 containing virtual object 

22 locations, and group assignment rules and retrieval plans. The signal may also provide for 

23 interactive virtual object requests from the reception site 30 to the interactive object service 

24 center 60 using delivery network 14 or the signal may be a means to provide access to the 

25 Internet or other public network through which virtual objects or content 36 are delivered. The 

26 signal sent over the PSTN maybe generatedby a data modem or transmission device, in which 
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1 the appropriate modem 3 102 is used to receive the signal and provide the embedded virtual 

2 objects to the modular connector 700 in the reception site 30 for processing. Alternatively, the 

3 reception site 30 may contain an internal modem 3 103, which receives the signal and provides 

4 the received signal with embedded virtual objects to the modular connector 700 in the 

5 reception site 30 for processing. 

6 Figure 24 presents embodiments associated with the delivery of virtual objects using 

7 wireless personal communications system (PCS) 320 1 to a reception site 30. Virtual objects 

8 are provided to the delivery network 1 1 by the object delivery center 1 5 or directly by the 

9 operations center 10. Alternatively, content 36 and virtual objects may be provided to the 

10 reception site 30 from the object delivery center 15 orfrom the local insertion center 20 using 

1 1 delivery network 12. The signal is then delivered over the PCS network 3201 delivery 

1 2 network. The wireless PCS system may be, for example a wireless LAN, digital cellular 

1 3 telephony network, analog cellular telephony network, digital cellular radio system, analog 

1 4 cellular radio system, digital pager network, analog pager network, or Personal Communication 

15 Network (PCN). The signal may provide for the delivery of virtual objects, content 36 

1 6 containing virtual object locations, and group assignment rules and retrieval plans. The signal 

17 may also provide for interactive virtual object requests from the reception site 30 to the 

1 8 interactive object service center 60 using delivery network 14 or the signal may be a means to 

1 9 provide access to the Internet or other public network through which virtual objects or content 

20 36 are delivered. A wireless PCS receiver 3202 is used to receive the signal and provide the 

2 1 embedded virtual objects to the modular connector 700 in the reception site 30 for processing. 

22 Alternatively, the reception site 258 may contain an internal wireless PCS receiver 3203 , which 

23 receives the signal and provides the received signal with embedded virtual objects to the 

24 modular connector 700 in the reception site 30 for processing. 

25 Figure 25 depicts several embodiments associated with the delivery of virtual objects 

26 using a national or local television broadcaster's signal. Virtual objects are provided to either 
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1 the national broadcaster 1110, the broadcast affiliate 1 1 12, or the local cable system 1 1 14 by 

2 the object delivery center 15 or directly by the operations center 10. The signal from the 

3 national broadcaster 1 1 1 0 can be delivered to reception site 30', 30* or 30"'using a satellite 

4 system 1 122, using a broadcast affiliate 1 1 12 terrestrially, or using alocal cable system 1 1 14. 

5 Alternatively, the local television broadcast affiliate 1 1 1 2 can originate the signal which can be 

6 delivered to the reception site 30', 30 w or 30 w terrestrially, or using a local cable system 1 1 14. 

7 The signal may provide for the delivery of virtual objects, content 36 containing virtual object 

8 locations, and group assignment rules and retrieval plans. The signal may also provide for 

9 interactive virtual object requests from the reception sites 30', 30" and 30"*to the interactive 

10 object service center 60 using delivery network 14 or the signal may be a means to provide 

1 1 access to the Internet or other public network through which virtual obj ects or content 36 are 

1 2 delivered. In one embodiment, the video signal is an analog video signal and the virtual objects 

1 3 is embedded in the video signal. In another embodiment, the video signal is a digital video 

14 signal and the virtual objects are carried as an independent data stream. In another 

1 5 embodiment, virtual objects may be embedded within the audio signal In one embodiment, 

1 6 the audio signal is an analog audio signal. In another embodiment, the audio signal is a digital 

17 audio signal. 

18 In yet another embodiment, the signal is a spread spectrum signal containing a digital 

1 9 data stream, requiring an appropriate spread spectrum receiver modular connector, such as the 

20 connector 700 of Figure 27, in the reception site 30', 30"or 30 w to extract the virtual objects. 

2 1 In this embodiment, the spread spectrum signal is transmitted in the same bandwidth as the 

22 video or audio signal, but below the noise level. 

23 Alternatively, several embodiments are associated with the delivery of virtual objects 

24 using a national or local radio broadcaster's signal. The signal from the national radio 

25 broadcaster can be delivered to the reception site 30', 30" or 30"'using the satellite system 

26 1122, orusingabroadcastaffiliate 1122. Alternatively, the radio broadcast affiliate 11 22 can 



-59- 



Docket 5220/PTO Klings/Spec.wpd 

1 originate the signal, which can be delivered to the reception site 30', 30*or 3(T, terrestrially. 

2 In one embodiment, the audio signal is an analog audio signal and the virtual objects is 

3 embedded in the audio signal. In another embodiment, the audio signal is a digital audio signal 

4 and the virtual objects are carried as an independent data stream. In yet another embodiment, 

5 the virtual objects are embedded in a sub-carrier of the analog audio broadcast. In another 

6 embodiment, the signal is a spread spectrum signal containing a digital data stream, requiring 

7 an appropriate spread spectrum receiver modular connector 700 in the reception site 30', 30" 

8 or 30"' to extract the virtual objects. In this embodiment, the spread spectrum signal is 

9 transmitted in the same bandwidth as the audio signal, but below the noise level. 

1 0 A local insertion center 20 or multiple local insertion centers may optionally be used 

1 1 to insert virtual objects into content 3 6 provided by an operations center 1 0 or another local 

1 2 insertion center 20, and any other content source. A local insertion center 20 may perform the 

1 3 same functions as an operations center 10. Figure 26 depicts a local insertion center 20. As 

14 shown in Figure 26, the local insertion center 20 includes a virtual object location definer 100', 

1 5 a virtual object selector 200', and a targeted virtual object management system 300' (TV OMS) 

1 6 which are identical to the virtual object location defmer 100, a virtual object selector 200, and 

17 a targeted virtual object management system 300 (TVOMS) of an operations center 10. A 

1 8 local insertion center 20 may detect existing virtual object locations in content 36 and replace 

1 9 existing virtual objects with new virtual objects, delete existing virtual objects, or add new 

20 virtual objects in existing virtual object locations and target the virtual objects to reception sites 

2 1 or groups of reception sites. Alternatively, a local insertion center 20 may create new virtual 

22 object locations and insert and target virtual objects within these new virtual object locations 

23 using the processes defined for the operations center 10. The local insertion center 20 may 

24 modify an existing or generate a new retrieval plan and modify an existing or generate new 

25 group assignment rules for distribution to reception sites. 
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1 Figure 27 depicts an example of a reception site 30 in more detail The modular 

2 connector 700 may handle all interactions with a reception site 30. Programming content 36 

3 with virtual obj ect locations and metadata packets containing placement guidelines, mattes , and 

4 retrieval plans are received by the reception site modular connector 700 and passed to the 

5 virtual object extractor processor 780. The virtual object extractor processor 780 removes 

6 any virtual obj ects from the received signal and the retrieval plan information and routes the 

7 virtual objects and retrieval plan to the storage management processor 710. The storage 

8 management processor 7 10 uses the retrieval plan to determine which virtual objects are 

9 destined to the reception site 30 and saves the required virtual objects in virtual object storage 

10 720. In an alternative embodiment, virtual objects may be received by the reception site 30 

1 1 independent of the programming content 36. Group assignment rules are received by the 

1 2 modular connector 700 and passed to the group assignment rules processor 735 . The group 

1 3 assignment rules processor 735 manages the reception site' s subscriber database. The group 

14 assignment rules processor 735 allows the subscriber to modify information within the 

1 5 subscriber database either directly or through on-screen menus and prompts. The group 

1 6 assignment rules processor 735 allows the subscriber to review the current group assignment 

17 rules in effect for the reception site 30. The group assignment rules processor 735 allows the 

1 8 subscriber to override any or all of the current group assignment rules in effect at the reception 

19 site or modify any or all group assignment rules in effect at the reception site 30. 

20 The programming content 36 with virtual object locations is then passed to the virtual 

2 1 object location detector processor 750. Information received about virtual object locations is 

22 extracted from the programming content 36 and passed to the selector processor 740 which 

23 coordinates with the storage management processor 7 1 0 to determine the appropriate virtual 

24 object 38 to place into each virtual object location 37 based on placement guidelines and 

25 available virtual objects stored in the virtual object storage 720. The storage management 

26 processor 7 10 retrieves the appropriate virtual object 38 for one or more virtual object location 
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1 37 contained in the content 36 from the virtual object storage 720. Virtual objects 

2 are passed from the storage management processor 710 to the virtual object insertion 

3 processor 760. 

4 Programming content 36 with virtual object locations is passed from the virtual object 

5 location detector processor 750 to the content buffer 790 where the programming content 36 

6 is stored for a fixed period of time and then played out of the content buffer 790 to the virtual 

7 object insertion processor 760. If a virtual object 38 is available for placement in a virtual 

8 object location 37, the virtual object 38 is inserted into the appropriate virtual object location 

9 37 by the virtual object insertion processor 760. 

10 In one embodiment, the virtual object location 37 may require that an embedded 

1 1 virtual object 38 be placed within the content 36. The virtual object insertion processor 760 

1 2 may use techniques for the insertion of embedded virtual objects which are described in detail 

13 in U.S . Patents 5 ,953,076, to Astle, Brian; and Das, Subhodev; titled System and Method of 

14 Real Time Insertions into Video Using Adaptive Occlusion with a Synthetic Reference Image; 

15 5,892,554, to DiCicco, Darrell; and Fant, Karl; entitled System and Method for Inserting 

1 6 Static and Dynamic Images into a Live Video Broadcast; 5,5 1 5,485, to Luquet, Andre; and 

17 Rebuffet, Michel; entitled Method and Device for Modifying a Zone in Successive Images; 

18 5,903,317, to Sharir, Avi; and Tamir, Michael; entitled Apparatus and Method for Detecting, 

19 Identifying and Incorporation Advertisements in a Video; and the MPEG4 standard, the 

20 disclosure of which are hereby incorporated by reference. 

21 In another embodiment, when the virtual object location 37 may require that an 

22 overlaid virtual object 38 be placed within the content 36. The virtual object insertion 

23 processor 760 may use techniques for the overlaying of virtual objects which are described in 

24 detail in U.S . Patents 4,3 1 9,266 to Bannister, Richard S.; entitled Chroma Keying System; 

25 4,999,709 to Yamazaki, Hiroshi; and Okazaki, Sakae; entitled Apparatus for Inserting Title 

26 Pictures; 5,249,039, to Chaplin, Daniel J. ; entitled Chroma Key Method and Apparatus; and 
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1 5,233,423 to Jernigan, Forest E.; and Bingham, Joseph; entitled Embedded Commercials 

2 within aTelevision Receiver using an Integrated Electronic Billboard, the disclosure of which 

3 are hereby incorporated by reference. Programming content 36 with embedded and overlaid 

4 virtual objects is passed to an optional interactive object processor 770. 

5 If an external trigger is received indicating the subscriber has selected an interactive 

6 virtual object 38, the interactive object processor 770 processes the request and passes the 

7 subscriber request to the modular connector 700. The modular connector 700, in turn, passes 

8 the request to the interactive object servicing center 60. The interactive object servicing center 

9 60 will process the request and may respond back to the reception site 30 with an interactive 

10 response. Preferably, when a virtual object 38 is placed into a virtual object location 37, the 

1 1 selector processor 740 records the event in the placement log 730. The placement log 730 

1 2 provides viewing data to the group assignment rules processor 735, where the information can 

1 3 be used for future virtual object targeting processing. The selector processor 740 can be 

14 provided targeting algorithm updates from external sources. 

15 An interactive object servicing center 60 is depicted in Figure 28. The interactive 

1 6 object servicing center 60 processes interactive requests and formulates responses to such 

17 requests. Figure 29 presents the process the interactive object servicing center 60 performs. 

18 Theprocessbeginswithblock4500. Inblock4501,theinteractiveobjectservicingcenter60 

19 receives interactive requests from reception sites. In block 4502, the interactive object 

20 servicing center 60 determines the appropriate action to be performed based on the received 

2 1 interactive request. In block 4503, the interactive object servicing center 60 performs the 

22 appropriate action based on the received interactive request. In block 4504, the interactive 

23 object servicing center 60 replies to the requesting reception site with an interactive response. 

24 As shown in Figure 28, communications to and from the interactive object servicing 

25 center 60 over a delivery network may be done using modular connector 700. An interface 

26 61 receives interactive requests from reception sites. The interface 61 can include a 
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1 workstation, such as the workstation 64, for example, from which an operator manually enters 

2 interactive request behavior for the interactive object servicing center 60. A processor 62 

3 processes the received interactive request, performs the appropriate action, retrieving 

4 information from database 63 to perform the actions and storing transaction information in 

5 database 63 to record the transaction event. 

6 A variety of virtual object targeting delivery systems have been described. One of 

7 ordinary skill in the art will recognize that the above description is that of preferred 

8 embodiments of the invention and the various changes and modification may be made thereto 

9 without departing from the spirit and scope of the invention as defined in the following claims. 
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1 What is claimed is: 

2 1 . A method for targeting virtual advertisements at a user' s terminal, comprising: 

3 assigning at least one virtual advertisement spot to a program; 

4 assigning one or more virtual objects to the at least one virtual advertisement 

5 spot; 

6 generating a retrieval plan; and 

7 providing the retrieval plan to the terminal, wherein the retrieval plan directs the 

8 terminal to select one of the one or more virtual objects. 

9 2. The method of claim 1, further comprising: 

10 defining one or more target categories; and 

1 1 for one or more of the target categories, designating one or more groups, wherein a 

12 group is based on common viewer characteristics. 

13 3 . The method of claim 2, further comprising assigning the terminal to one or more of the 

14 groups, comprising: 

1 5 generating group assignment rules; 

16 providing the group assignment rules to the terminal; 

17 storing the group assignment rules at the terminal; and 

1 8 determining one or more group assignments based on the group assignment rules and 

19 data related to the terminal. 

20 4. The method of claim 3, wherein the data related to the terminal includes one or more 

2 1 of Area of Dominant Influence (ADI), zip code+4, demographic data and programs watched 

22 data, virtual objects viewed, on-screen questionnaires and characteristics imported from 

23 marketing databases, the group assignments being updated to reflect changes in the ADI, zip 



-65- 



Docket 5220/PTO Filings/Spec.wpd 



1 code+4, demographic data, programs watched data, virtual objects viewed, on-screen 

2 questionnaires, and characteristics imported from marketing databases. 

3 5. The method of claim 2, wherein generating the retrieval plan comprises: 

4 designating a unique group mask for one or more of the groups; and 

5 assigning one or more of the groups to one of the virtual objects, wherein the group 

6 mask indicates whether the terminal displays a particular virtual object. 

7 6. The method of claim 2, wherein the retrieval plan and the group assignment rules are 

8 sent periodically to the terminal. 

9 7. A method of targeting virtual objects, comprising: 

10 providing a program containing one or more virtual object locations; 

1 1 providing virtual objects for one or more of the virtual object locations; 

1 2 providing alternate virtual objects for one or more of the virtual object locations; and 

1 3 generating a retrieval plan at one or more viewer' s terminals, wherein the retrieval plan 

14 designates which of the one or more virtual object locations displays an alternate virtual object. 

15 8. The method of claim 7, wherein the program is a television program. 

16 9 . The method of claim 7 , wherein the program is one of an advertisement, an electronic 

17 program guide, and an Internet web page. 

18 10. The method of claim 7, wherein at least one of the virtual object locations is fixed in 

19 position across frames of the program. 
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1 11. The method of claim 7, wherein at least one of the virtual object locations moves 

2 spatially in the program with time. 

3 12. The method of claim 7, wherein at least one of the virtual objects is interactive. 

4 13. The method of claim 7 , wherein the program is broadcast to the terminals, further 

5 comprising: 

6 creating categories of virtual objects and programs; 

7 within one or more of the categories, defining one or more groups; 

8 generating group assignment rules based on common viewer characteristics; 

9 providing the group assignment rules to one or more of the terminals; 

10 storing the group assignment rules in one or more of the terminals; 

1 1 using the stored group assignment rules, assigning one or more of the terminals to one 

12 or more of the groups; and 

1 3 comparing the retrieval plan to the group assignments to determine virtual objects to 

14 display in the one or more virtual object locations. 

15 14. The method of claim 13, wherein generating the retrieval plan, comprises: 

16 assigning the virtual objects to the one or more virtual object locations; 

17 assigning the alternate virtual objects to at least one of the one or more virtual object 

18 locations; 

1 9 assigning a group to one or more of the virtual objects and the alternate virtual objects; 

20 and 

2 1 creating a group mask assignment, wherein the group mask assignment is used at 

22 the terminals to compare the retrieval plan to the assignments of the terminals to the 

23 groups. 
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1 15. The method of claim 14, wherein assigning the group to the virtual objects and the 

2 alternate virtual objects, comprises: 

3 ranking one or more programs based on the categories of targeted virtual objects and 

4 a first percentage of total viewers who may view the one or more programs; 

5 ranking the targeted virtual objects based on a second percentage of the total viewers; 

6 determining, for the one or more ranked programs and the categories of targeted 

7 virtual objects, targeted virtual objects with overall highest rankings, based on the first and the 

8 second percentages; 

9 assigning targeted virtual objects with the overall highest rankings to be displayed as 

10 the virtual objects; and 

1 1 assigning targeted advertisements with lower overall rankings to be displayed as the 

12 alternate virtual objects. 

13 16. The method of claim 13, wherein the common viewer characteristics include viewer 

14 demographic information. 

15 17. The method of claim 13, wherein the common viewer characteristics include viewer- 

1 6 entered information. 

17 18. The method of claim 13, wherein the common viewer characteristics include programs 

18 watched data. 

19 19. The method of claim 1 3, wherein the common viewer characteristics include virtual 

20 objects watched data. 
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1 20. The method of claim 13, wherein one or more ofthe virtual object locations contain 

2 an interactive virtual obj ect, and wherein the common viewer characteristics include viewer 

3 activation of the interactive virtual object. 

4 21 . The method of claim 13, wherein the terminals are television set top terminals. 

5 22. The method of claim 13, wherein one or more ofthe terminals is incorporated into one 

6 of a television, a personal computer and a PDA with video viewing capabilities. 

7 23. The method of claim 13, wherein one or more ofthe terminals is a satellite television 

8 receiver. 

9 24. The method of claim 13, further comprising: 

1 0 at one or more ofthe terminals, recording in a memory an identification of a virtual 

1 1 object displayed in a virtual object location; 

12 providing the identification to a remote site; and 

1 3 deleting the identification from the memory. 

14 25. A method of targeting virtual objects to terminals, 

15 comprising: 

16 creating a package of targeted virtual objects; 

17 providing the package to one or more of the terminals; 

1 8 generating a retrieval plan; 

19 storing the retrieval plan at one or more of the terminals; and 
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1 providing a program to one or more of the terminals, the program including at least 

2 one virtual object location, wherein the retrieval plan designates virtual objects to be displayed 

3 during a display of the program. 

4 26. The method of claim 25, further comprising: 

5 generating group assignment rales; 

6 storing the group assignment rules at one or more of the terminals; and 

7 creating group assignments by assigning one or more of the terminals to one or more 

8 groups based on the group assignment rules. 

9 27. The method of claim 26, further comprising at one or more of the terminals receiving 

10 the program, retrieving one of the targeted virtual objects for display in the at least one virtual 

11 object location. 

12 28. The method of claim 27, wherein the retrieval step, comprises: 

13 comparing the group assignments to the retrieval plan; and 

14 selecting a virtual object for display based on the comparison. 

1 5 29. The method of claim 26, wherein one or more of at least one virtual object locations 

16 contains an interactive virtual object, further comprising: 

17 receiving a selection of the interactive virtual object; and 

18 linking a terminal selecting the interactive virtual object to an alternate 

19 program. 

20 30. The method of claim 29, wherein the alternative program comprises an Internet web 

21 site. 
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1 31. The method of claim 26, wherein the step of creating the group assignments comprises 

2 analyzing individual terminal data and terminal group data. 

3 32. The method of claim 3 1 , wherein the individual terminal data comprises one or more 

4 of viewer demographic data, programs watched data, virtual objects viewed data, on-screen 

5 questionnaires, and characteristics imported from marketing databases, and wherein the 

6 terminal group data, comprises one or more of ADI, zip code, and geographical data. 

7 33 . The method of claim 3 1 , wherein one or more of the terminals comprises a global 

8 positioning satellite receiver, further comprising: 

9 determining a geographical location of one or more of the terminals; and 

1 0 storing the geographical location of the one or more of the terminals as individual 

1 1 terminal data. 



12 34. A method for assigning targeted virtual objects to virtual object locations in one or 

13 more programs, comprising: 

14 identifying the one or more programs to carry the targeted virtual objects; 

15 assigning the targeted virtual objects to target categories; 

16 ranking one or more of the programs based on the target categories and a first 

17 percentage of total viewers in one or more groups of viewers; 

1 8 ranking the targeted virtual objects based on a second percentage of total viewers in 

19 the one or more groups of viewers; 

20 determining, for one or more of the programs and one or more of the target 

2 1 categories, targeted virtual objects with overall highest rankings, based on the first and the 

22 second percentages; 
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1 assigning one or more targeted virtual objects as default virtual objects; 

2 assigning one or more targeted virtual objects as alternate virtual objects; and 

3 assigning the default virtual objects and the alternate virtual objects to the virtual object 

4 locations. 

5 35. The method of claim 34, wherein the first and second percentages of total 

6 viewers are based on viewer demographic information including zip code, ADI and 

7 geographical data. 

8 36. The method of claim 34, wherein assigning the default virtual objects and the alternate 

9 virtual objects, comprises: 

10 dividing one or more target categories into groups; 

1 1 generating group assignment rules that are used to assign or more of the terminals to 

12 one or more of the groups; 

1 3 storing the group assignment rules at one or more of the terminals, wherein processors 

14 at the one or more terminals assign the terminals to one or more groups using the group 

15 assignment rules; 

16 generating a retrieval plan; and 

17 providing the retrieval plan to one or more of the terminals receiving the 

18 programs at the one or more terminals, comparing the group assignments to the retrieval 

19 plan. 

20 37. The method of claim 36, further comprising: 

21 revising the retrieval plan and the group assignment rules; and 

22 providing the revised retrieval plan and the revised group assignment rules to the one 

23 or more terminals . 
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1 38. The method of claim 36, wherein the retrieval plan and the group assignment rules are 

2 provided over an Internet. 

3 39. The method of claim 34, wherein the package of targeted virtual objects is provided 

4 to a terminal over an Internet. 

5 40. The method of claim 34, wherein the package of targeted virtual objects is provided 

6 to a terminal using one of a public switched telephone network, a cable television network, a 

7 satellite television network, a local area network and a fiber optic network. 
8 

9 41. The method of claim 34, wherein virtual objects are provided with program content. 

10 42. The method of claim 34, wherein virtual objects are provided independently of 

1 1 program content. 
12 

13 43 . A method for targeting virtual objects to subscribers in a television program delivery 

14 system, comprising: 

1 5 gathering information related to one or more of subscribers, wherein individual 

16 subscriber's information is gathered and stored at individual subscriber's terminals; 

1 7 analyzing the gathered information to determine a subscriber profile for one or more 

18 of the subscribers; 

1 9 correlating the subscriber profile with categories of virtual objects, wherein one or 

20 more virtual object categories includes at least one virtual object; and 

21 selecting from the correlated virtual object for display based on the correlation. 

22 44. The method of claim 43 , wherein the individual subscriber' s information includes 

23 virtual objects watched data. 
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1 45 . The method of claim 43, further comprising defining virtual object locations, wherein 

2 the virtual objects are displayed in the virtual object locations. 

3 46 . The method of claim 45, wherein the virtual object locations are defined in a television 

4 program. 

5 47. The method of claim 45, wherein the virtual object locations are defined in an 

6 advertisement provided over a television delivery system. 

7 48. An apparatus that targets virtual objects for display at a viewer reception site, 

8 comprising: 

9 a virtual object location identifier that identifies virtual object locations in a 

10 video; 

1 1 a virtual object selector coupled to the virtual object location identifier that selects 

12 one or more virtual objects to display in one or more of the identified virtual object 

13 locations; 

14 a targeted virtual object manager coupled to the virtual object selector, wherein the 

1 5 targeted virtual object manager determines virtual objects from the selected one or more virtual 

16 objects to be displayed in one or more of the virtual object locations; and 

17 a group assignment rules generator that provides group assignment rules for classifying 

1 8 the reception site. 

19 49. The apparatus of claim 48, wherein the determined virtual objects are determined 

20 based on the group assignment rules and a personal profile of a viewer of the reception site, 

21 and wherein the personal profile is stored in a memory of the reception site. 
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1 50. The apparatus of claim 49, wherein the personal profile is based on one or more of 

2 programs watched data, virtual objects viewed data and other viewer data collected and stored 

3 at the reception site. 

4 51. A routine, executable on a general purpose computer, for targeting virtual obj ects to 

5 an individual viewer, the routine, comprising: 

6 a group definition routine that defines groups based on common viewer characteristics; 

7 a group assignment routine that assigns individual viewer terminals to one or more of 

8 the groups; 

9 a virtual object location routine that determines available virtual object locations in a 

10 program; and 

1 1 a retrieval plan generator that generates a plan for retrieving one or more virtual 

1 2 objects for display in one or more of the available virtual object locations, wherein the retrieval 

13 plan is generated based on the group definitions. 

14 52. A method for targeting virtual objects to viewers, comprising: 

15 recognizing a virtual object location in a program; 

16 receiving one or more virtual objects; and 

1 7 generating a retrieval plan that directs viewers' terminals to insert one or more of the 

18 virtual objects into one or more of the virtual object locations. 

19 53. A method for targeting virtual objects to locations in a program, comprising: 

20 at a local viewer terminal, identifying virtual objects for insertion into one or 

21 more of the locations; 

22 generating an individual viewer profile; and 
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providing one or more of the identified virtual objects for insertion based on the 
individual viewer profile. 



3 54. A method for targeting virtual objects to terminals, comprising: 

4 identifying the terminals based on characteristics of individual terminals; 

5 identifying virtual object locations in programs for display at the terminals; and 

6 targeting the virtual objects for insertion into the virtual object locations based 

7 on the identities of the terminals. 



8 55 . The method of claim 54, further comprising displaying multiple virtual objects 

9 simultaneously 

10 56. The method of claim 54, wherein one or more of the virtual objects are interactive 

1 1 virtual objects, further comprising: 

1 2 receiving a selection of one or more of the interactive virtual objects from one or more 

13 of the terminals; 

14 linking the selecting terminal to a remote location; and 

15 displaying content from the remote location at the selecting terminal. 

16 57. The method of claim 56, wherein the remote location is an operations center and the 

17 content is an additional program. 

18 58. The method of claim 56, wherein the remote location is an Internet web site and the 

19 content is one or more web pages. 
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1 59. A terminal in a television program delivery system that targets virtual objects for 

2 display to a viewer of the terminal, comprising: 

3 a receiver that receives programs containing virtual obj ect locations, virtual objects, 

4 and a virtual object retrieval plan; 

5 a memory that stores the virtual object retrieval plan; and 

6 a processor that executes a group assignment routine to assign the terminal to one or 

7 more groups, wherein the groups are defined based on common viewer characteristics, and 

8 executes a virtual object assignment routine that assigns virtual objects to the virtual object 

9 locations based on a comparison of the retrieval plan and the group assignments. 

1 0 60. The terminal of claim 59, wherein the group assignment rules include an initial set of 

1 1 group assignment rules stored in the memory. 

12 61. The terminal of claim 59, wherein the receiver receives and the memory stores group 

1 3 assignment rules, and wherein the processor uses the group assignment rules and data related 

14 to the terminal to execute the group assignment routine. 

1 5 62. The terminal of claim 6 1 , wherein the data related to the terminal is stored in the 

1 6 memory, the data related to the terminal including internal information generated internally at 

1 7 the terminal and external information generated external to the terminal and provided to the 

18 terminal. 

19 63 . The terminal of claim 62, wherein the terminal further comprises a global positioning 

20 satellite (GPS) receiver, the GPS receiver determining a location of the terminal, wherein the 

2 1 location is stored in the memory as internal information, and wherein the processor uses the 

22 location to determine area of dominant influence and postal code information. 
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1 64 . The terminal of claim 62, wherein the internal information includes programs watched 

2 information and virtual objects viewed information for programs and virtual objects displayed 

3 at the terminal. 

4 65 . The terminal of claim 6 1 , wherein the group assignment rules are periodically revised, 

5 the memory storing the revised group assignment rules and the processor determining revised 

6 group assignments based on the revised group assignment rules. 

7 66. The terminal of claim 6 1 , wherein the data related to the terminal is revised, the 

8 processor determining revised group assignments based on the revised data related to the 

9 terminal. 

10 67. The terminal of claim 61, wherein the processor uses the group assignment 

1 1 rules to derive terminal identifying information based on other information stored at the 

12 terminal. 

13 68 . A method for assigning targeted virtual objects to virtual object locations in one or 

14 more programs, comprising: 

15 identifying the one or more programs to carry the targeted virtual objects; 

1 6 assigning the targeted virtual objects to target categories and to groups within the 

17 target categories; 

18 generating virtual object location group percentage breakdowns, wherein the 

1 9 generating step comprises using generic viewing population information to assign the groups to 

20 the one or more programs; 

2 1 creating virtual object ranking percentages based on the generic viewing population 

22 information; 
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1 determining, for one or more of the programs and one or more of the target 

2 categories, targeted virtual objects with overall highest ranking percentages and group 

3 percentage breakdowns; and 

4 assigning virtual objects to the virtual object locations based on the determined ranking 

5 percentages and group percentage breakdowns. 

6 69. A method of targeting virtual objects to terminals, comprising: 

7 identifying virtual objects; 

8 providing the one or more of the virtual objects to one or more of the terminals; 

9 generating a retrieval plan; 

10 storing the retrieval plan at one or more of the terminals; and 

1 1 providing a program to one or more of the terminals, the program including at least 

1 2 one virtual object location, wherein the retrieval plan designates virtual objects to be displayed 

13 during a display of the program. 

14 70. The method of claim 69, further comprising: 

1 5 generating group assignment rules; 

16 storing the group assignment rules at one or more of the terminals; and 

1 7 creating group assignments by assigning one or more of the terminals to one or more 

18 groups based on the group assignment rules. 

19 71. The method of claim 70, wherein the step of generating group assignment rules 

20 comprises generating initial group assignment rules, and wherein the step of storing the group 

21 assignment rules comprises storing the initial group assignment rules. 
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1 72. The method of claim 7 1 , wherein the initial group assignment rules are stored at or 

2 before an initial use of a terminal. 

3 73 . The method of claim 70, further comprising at one or more of the terminals receiving 

4 the program, retrieving one of the targeted virtual objects for display in the at least one virtual 

5 object location. 

6 74. The method of claim 73, wherein the retrieval step, comprises: 

7 comparing the group assignments to the retrieval plan; and 

8 selecting a virtual object for display based on the comparison. 

9 75 . The method of claim 70, wherein one or more of at least one virtual object locations 

10 contains an interactive virtual object, further comprising: 

1 1 receiving a selection of the interactive virtual object; and 

12 linking a terminal selecting the interactive virtual object to an alternate 

13 program. 

14 76. The method of claim 70, wherein the group assignment rules are provided to aterminal 

1 5 using one of a public switched telephone network, a cable television network, a satellite 

16 television network, a local area network, a fiber optic network and an Internet 

17 77. The method of claim 69, wherein the virtual objects are provided to a terminal using 

1 8 one of a public switched telephone network, a cable television network, a satellite television 

19 network, a local area network, a fiber optic network and an Internet. 
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7 8 . The method of claim 69, wherein the retrieval plan is provided to a terminal using one 
of a public switched telephone network, a cable television network, a satellite television 
network, a local area network, a fiber optic network and an Internet. 
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1 ABSTRACT 

2 A method and an apparatus are used to target virtual objects to subscribers in a 

3 television delivery system. Programs are selected and virtual object locations are defined in the 

4 selected programs. The virtual objects available for targeting are categorized and the 

5 categories are correlated to viewing population information to determine the optimum targeting 

6 of the virtual objects. The virtual objects may be targeted based on information related to 

7 groups of subscribers. When a frame of a program includes a virtual object location, a default 

8 or an alternate virtual object is displayed. The virtual object location may change over space 

9 or time. The virtual object may be interactive, and may be used to link a subscriber to a remote 

10 location, such as an Internet web site. An operations center or a cable headend may generate 

1 1 group assignment rules that are stored in individual terminals and are used to assign the 

12 subscribers' television terminals to groups, based on factors such as area of dominant influence 

13 and household income maintained securely and privately within the terminal. A retrieval plan 

14 is then generated that instructs the television terminals to select the desired virtual object for 

1 5 display. The television terminals record which virtual objects were displayed, and may use this 

16 information to assist in targeting the virtual object. 
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